Information processing apparatus, and method

ABSTRACT

An information processing apparatus includes a processor and a memory with instructions thereon. The instructions upon execution by the processor, causes the processor to: sequentially sense a movement of a body of a user; present, to the user, an avatar corresponding to the user, rather than a photographed image of an appearance of the user, based on a result of the sensing so that the user perceives a shape of a surface of the body of the user; determine a position of a figure object to be drawn that serves as a guide for movement of the body of the user in association with the avatar based on the result of the sensing; and display the figure object at the determined position.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation application of No. PCT/JP2021/026675,filed on Jul. 15, 2021, and the PCT application is based upon and claimsthe benefit of priority from Japanese Patent Application No.2020-136049, filed on Aug. 11, 2020, Japanese Patent Application No.2021-063763, filed on Apr. 2, 2021, Japanese Patent Application No.2021-063771, filed on Apr. 2, 2021, and Japanese Patent Application No.2021-096232, filed on Jun. 8, 2021, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,and a method.

BACKGROUND ART

There are known techniques for taking a video of a user moving with acamera and evaluating the movement of the user based on the taken video.

In Japanese Patent Laid-Open No. 2016-81504, there is described atechnique relating to an apparatus for judging the pros and cons of rankpromotion by recognizing the form of a martial art based on the skeletonof the human body with a depth camera and automatically evaluating themovement of the user.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a configuration of the whole of ameasurement system 1.

FIG. 2 is a block diagram showing a terminal apparatus 10 forming themeasurement system 1 according to the first embodiment.

FIG. 3 is a diagram showing a functional configuration of a server 20.

FIG. 4 is a diagram showing data structures of a user informationdatabase 281, a measurement item information 282, a measurement historydatabase 283 and a friend list 284 stored in the server 20.

FIG. 5 shows a situation where users measure movements of their bodies.

FIG. 6 is a diagram showing details of a mat 32.

FIG. 7 is a flowchart showing a process in which the terminal apparatus10 performs a ROM check and updates a measurement history.

FIG. 8 is a flowchart showing a process in which the terminal apparatus10 evaluates a joint range of motion by measuring the movement of theuser stepping his or her feet forward.

FIG. 9 is a flowchart showing a process in which the terminal apparatus10 evaluates a joint range of motion by measuring the movement of theuser twisting the upper part of the body.

FIG. 10A is a diagram showing examples of details of notifications onthe terminal apparatus 10 in a situation where the movement of the bodyof the user is measured.

FIG. 10B is a diagram showing examples of details of notifications onthe terminal apparatus 10 in a situation where the movement of the bodyof the user is measured.

FIG. 10C is a diagram showing examples of details of notifications onthe terminal apparatus 10 in a situation where the movement of the bodyof the user is measured.

FIG. 10D is a diagram showing examples of details of notifications onthe terminal apparatus 10 in a situation where the movement of the bodyof the user is measured.

FIG. 11 shows a situation where the measurement system 1 according to asecond embodiment measures the movement of the user walking.

FIG. 12 shows parts of the body of the user identified by the terminalapparatus 10 based on the measurement result from a measurementapparatus 31.

FIG. 13 is a diagram showing data structures of a measurement resultdatabase 287 and an evaluation result database 288 stored in the server20 according to the second embodiment.

FIG. 14 is a flowchart showing a process in which the terminal apparatus10 measures and evaluates a walking movement of the user in the secondembodiment.

FIG. 15A is a diagram showing example outputs in a case where theterminal apparatus 10 outputs a measurement result of the walkingmovement of the user and an evaluation result of the walking movement ofthe user based on the measurement result in the second embodiment.

FIG. 15B is a diagram showing example outputs in a case where theterminal apparatus 10 outputs a measurement result of the walkingmovement of the user and an evaluation result of the walking movement ofthe user based on the measurement result in the second embodiment.

FIG. 15C is a diagram showing example outputs in a case where theterminal apparatus 10 outputs a measurement result of the walkingmovement of the user and an evaluation result of the walking movement ofthe user based on the measurement result in the second embodiment.

FIG. 16 is a diagram showing an example output in a case where theterminal apparatus 10 outputs a measurement result of the walkingmovement of the user and an evaluation result of the walking movement ofthe user based on the measurement result in the second embodiment.

FIG. 17A is a diagram showing examples where the terminal apparatus 10outputs an evaluation result of the load on the knees in the walkingmovement of the user. This corresponds to step S1409 in FIG. 14.

FIG. 17B is a diagram showing examples where the terminal apparatus 10outputs an evaluation result of the load on the knees in the walkingmovement of the user. This corresponds to step S1409 in FIG. 14.

FIG. 18 is a flowchart showing a process in which the terminal apparatus10 measures and evaluates the balance of the trunk of the user in athird embodiment.

FIG. 19A is a diagram showing examples in which the terminal apparatus10 outputs a measurement result and an evaluation result of the balanceof the trunk of the user in the third embodiment.

FIG. 19B is a diagram showing examples in which the terminal apparatus10 outputs a measurement result and an evaluation result of the balanceof the trunk of the user in the third embodiment.

FIG. 19C is a diagram showing examples in which the terminal apparatus10 outputs a measurement result and an evaluation result of the balanceof the trunk of the user in the third embodiment.

FIG. 20 is a diagram showing data structures of various data stored inthe server 20 in the third embodiment.

FIG. 21A shows example screens in a situation where the terminalapparatus 10 presents a possible future trouble of the body of the userand proposes an exercise regimen corresponding to the trouble based onthe measurement result and evaluation result of the movement of the bodyof the user.

FIG. 21B shows example screens in a situation where the terminalapparatus 10 presents a possible future trouble of the body of the userand proposes an exercise regimen corresponding to the trouble based onthe measurement result and evaluation result of the movement of the bodyof the user.

FIG. 22 shows a flow of a process performed by the apparatuses in afourth embodiment.

FIG. 23 is a diagram showing data structures of various data stored inthe server 20 in an example modification of the fourth embodiment.

FIG. 24 is a diagram showing a flow of an operation in the examplemodification of the fourth embodiment.

FIG. 25A shows example screens for the user to respond a questionnairein a situation where the terminal apparatus 10 presents a possiblefuture trouble of the body of the user based on the measurement resultand evaluation result of the movement of the body of the user, andproposes an exercise regimen corresponding to the trouble.

FIG. 25B shows example screens for the user to respond a questionnairein a situation where the terminal apparatus 10 presents a possiblefuture trouble of the body of the user based on the measurement resultand evaluation result of the movement of the body of the user, andproposes an exercise regimen corresponding to the trouble.

FIG. 26 is a diagram showing data structures of a trainee informationdatabase 2601, a trainer information database 2602, and a traininghistory database 2603 stored in the server 20 in a fifth embodiment.

FIG. 27 is a flowchart showing a series of processing in which themovement of the body of a first user is sensed as point cloud data, andinformation based on the point cloud data on the first user obtained bythe sensing is transmitted to a terminal of a second user.

FIG. 28 is a flowchart showing a flow of a process in which a terminalof a trainer receives a voice input from the trainer and transmits thevoice input to a terminal of a trainee.

FIG. 29 is a flowchart showing a process in which the movement of thebody of the first user is sensed as point cloud data, and an instructionto the first user is identified based on information on the point clouddata on the first user obtained by the sensing and a learned model.

FIG. 30 shows an example screen when a terminal apparatus 10A of thefirst user senses the movement of the body of the first user as pointcloud data and transmits information based on the point cloud data onthe first user obtained by the sensing to the terminal of the seconduser.

FIG. 31 shows example screens for the first user and the second user andan overall arrangement of the system when the movement of the body ofthe first user is sensed as point cloud data, and information based onthe point cloud data on the first user obtained by the sensing istransmitted to the terminal of the second user.

FIG. 32 shows an example screen when the movements of bodies of aplurality of users are sensed as point cloud data, and information basedon the point cloud data on the users obtained by the sensing istransmitted to a defined virtual space.

FIG. 33 is a diagram showing a concept of a process in which theterminal apparatus 10A draws an auxiliary line based on the result ofestimation of the posture of the user.

FIG. 34 shows an example screen presented to the user.

FIG. 35 is a diagram showing an example screen for the user in astanding posture to adjust a forward, backward, left or right leaning ofthe body, displayed on the terminal apparatus 10A.

FIG. 36 is a diagram showing a process of determining the position of afigure object to be drawn that serves as a guide for movement of thebody of the user based on the result of estimation of the position of aparticular part of the user.

FIG. 37 is a diagram showing an example screen for the terminalapparatus 10A to prompt the user to make a movement of lateral armelevation.

FIG. 38 is a diagram showing a process of determining the position of afigure object to be drawn that serves as a guide for movement of thebody of the user based on the result of estimation of the position of aparticular part of the user.

FIG. 39 is a diagram showing a course of movement of the body of theuser.

FIG. 40 is a diagram showing a state of the user having completelyraised both arms at the timing of a peak of the movement of lateral armelevation.

FIG. 41 is a diagram showing an example in which a result of sensing ofthe user at the timing of the peak of the movement of lateral armelevation is saved and displayed as an avatar for comparison.

FIG. 42 is a diagram showing an example screen for the terminalapparatus 10A to prompt the user to make a squat movement.

FIG. 43 is a diagram showing a process of determining the position of afigure object to be drawn that serves as a guide for movement of thebody of the user based on the result of estimation of the position of aparticular part of the user.

FIG. 44 is a diagram showing a course of movement of the body of theuser.

FIG. 45 is a diagram showing a state of the user having completelysquatted down at the timing of a peak of the squat movement of the user.

FIG. 46 is a diagram showing an example in which a result of sensing ofthe user at the timing of the peak of the squat movement is saved anddisplayed as an avatar for comparison.

FIG. 47 is a diagram showing an example screen for the terminalapparatus 10A to prompt the user to make a movement of lowering his orher hips while stepping a foot forward.

FIG. 48 is a diagram showing an example screen in a situation where itis indicated to the user that the initial posture of the user is proper.

FIG. 49 is a diagram showing a situation where the measurement of themovement of the user is started.

FIG. 50 is a diagram showing a situation where whether the movement ofthe user is proper or not is determined, and the determination result ispresented to the user.

FIG. 51 is a diagram showing a situation where whether the movement ofthe user is proper or not is determined, and the determination result ispresented to the user.

DESCRIPTION OF EMBODIMENTS

In the following, embodiments of the present disclosure will bedescribed with reference to the drawings. In the following description,the same components are denoted by the same reference symbols. The samecomponents have the same names and same functions. Therefore, detaileddescription thereof will not be repeated.

According to an embodiment, an information processing apparatus isprovided. The apparatus comprises a processor and a memory withinstructions thereon. The instructions upon execution by the processor,causes the processor to: sequentially sense a movement of a body of auser; present, to the user, an avatar corresponding to the user, ratherthan a photographed image of an appearance of the user, based on aresult of the sensing so that the user perceives a shape of a surface ofthe body of the user; determine a position of a figure object to bedrawn that serves as a guide for movement of the body of the user inassociation with the avatar based on the result of the sensing; anddisplay the figure object at the determined position.

First Embodiment

<Overview>

In the embodiment described below, a measurement system 1 will bedescribed which evaluates the flexibility of the body of a user bymeasuring the balance of the body of the user or a movement of the bodyof the user, and provides an exercise regimen for the user to achieve agood posture (such as a good walking posture).

If the walking posture of a user is not good, the user may make anunnecessary movement while walking, so that the user may unnecessarilyconsume his or her energy and easily get tired. If the user achieves agood walking posture, the user can make functional movements and as aresult can walk with less effort. In view of this, the measurementsystem 1 (1) measures the movement of the user while walking to evaluatethe walking posture (whether the user is properly walking in awell-balanced manner or not), (2) measures the flexibility of a joint ofthe user to perform the range of motion (ROM) check that evaluates thejoint range of motion, and (3) evaluates the condition of the muscles ofthe user and provides a training regimen.

When the user is walking in a well-balanced manner (for example, themovement of the body of the user is not off-balance to the left or rightor the movement of the muscles while walking is ideal), the user ismaking functional movements. The functional movement means that the useris using his or her muscles for walking without unnecessarily consumingenergy. That is, when the user uses his or her muscles to providepropulsion for walking, the movement of the muscles efficiently providesthe propulsion. As a result, the user can give an impression that theuser is walking in a beautiful posture. In order for the user to achievesuch a well-balanced waking posture, it is useful to evaluate the jointrange of motion. The measurement system 1 evaluates the joint range ofmotion and provides the user with the evaluation result of the jointrange of motion and a strength training regimen for achieving a goodwalking posture based on the evaluation result of the joint range ofmotion.

As described above, the measurement system 1 measures (checks) themovement of the body of the user and provides a body exercise for theuser.

The measurement system 1 can be installed in an office of a businesscompany, a commercial facility, a shop space, a studio, or an ordinaryhousehold, for example. A business company may introduce the measurementsystem 1 in order to improve the health of the employees, for example. Ashop in a commercial facility may use the measurement system 1 toprovide health care services to customers.

<1. Diagram Showing Configuration of Whole of System>

FIG. 1 is a diagram showing a configuration of the whole of themeasurement system 1.

As shown in FIG. 1, the measurement system 1 includes a plurality ofmeasurement units including various sensors that measure a movement ofthe body of a user (in the example shown, measurement units 30A, 30B,and 30C are shown. In the following, the measurement unit may bereferred to also as a measurement unit 30), a terminal apparatus 10 thatcontrols the operation of the measurement units (in the example shown, aterminal apparatus 10 is shown. In the following, the terminal apparatusmay be referred to also as a terminal apparatus 10A, a terminalapparatus 10B or the like when there are terminal apparatuses of aplurality of users. The same holds true for a component (such as aprocessor) of the terminal apparatus), and a server 20 that storesmeasurement results of movements of the body of the user.

The measurement unit 30 is connected wirelessly or by wire to theterminal apparatus 10. As shown in the drawing, a plurality ofmeasurement units 30 may be connected to one terminal apparatus 10, or aterminal apparatus 10 may be connected to each of a plurality ofmeasurement units 30 (that is, one terminal apparatus 10 may be preparedfor each measurement unit 30 for user measurement).

The measurement unit 30 includes a measurement apparatus (in the exampleshown, measurement apparatuses 31A, 31B, and 31C are shown. These may begenerically referred to as a measurement apparatus 31), a mat (in theexample shown, mats 32A, 32B, and 32C are shown. These may begenerically referred to as a mat 32.), and a pole (in the example shown,poles 33A, 33B, and 33C are shown. These may be generically referred toas a pole 33.).

The measurement apparatus 31 is intended to measure a movement of thebody of a user. For example, the measurement apparatus 31 detects thedistance and direction to a measurement target with an RGB-D camera (adepth sensor). The measurement apparatus 31 may further have amicrophone array or the like to detect a voice of a user or thedirection from which the voice comes, for example. For example, themeasurement apparatus 31 may be a device capable of detecting a gestureof a user or receiving an operation of a user through voice recognition,such as Kinect (registered trademark). The measurement apparatus 31detects the body of a user, thereby detecting the position of the userand the positions of joints of the body of the user. In this way, themeasurement apparatus 31 sets virtual bones that virtually connectjoints of the user, thereby measuring the movement of the body of theuser. The measurement apparatus 31 may be other terminals than Kinect(registered trademark), such as a smartphone. For example, when asmartphone is provided with a depth camera, point cloud datarepresenting the body of the user can be obtained with the smartphone.

The measurement apparatus 31 need not have a function of measuringdistance and may obtain the distance to a subject (the distance to ajoint of the subject, for example) based on a taken image by locatingthe subject based on the taken image and using a leaned model thatestimates the distance between the subject and the camera. Even when themeasurement apparatus 31 cannot measure distance (such as when themeasurement apparatus 31 is an RGB camera), the measurement apparatus 31may locate parts of the subject based on the taken image and estimatethe skeleton (bones). A plurality of cameras may be provided as themeasurement apparatus 31, and the distance to the subject may beestimated from images taken from different viewpoints.

The mat 32 provides a guide for the foot positions of the user formeasurement of the movement of the body of the user. The mat 32 isdisposed in a predetermined positional relationship to the measurementapparatus 31. That is, the mat 32 is disposed within a range where themeasurement apparatus 31 can measure the movement of the body of theuser. As described later, the mat 32 includes a plurality of marks as aguide for the foot positions of the user. The user adjusts the footpositions or makes a movement of stepping forward by watching the marks.

The pole 33 is used with the mat 32 to measure the movement of the bodyof the user, such as a movement of the user twisting his or her body.The pole 33 is disposed on the mat 32 in a position associated with theguide marks for the foot positions of the user. With both the feet incontact with the mat 32 (that is, with the foot positions keptunchanged), the user makes a movement of twisting his or her body totouch any of a pair of poles placed on either side of the mat 32.Therefore, the pole 33 is used as a target point to which the arms aremoved in order for the user to make the body twisting movement. The pole33 may be set at an appropriate height for the user so that the user cangrasp the pole 33 with his or her hands when twisting his or her body.

A combination of the measurement apparatus 31, the mat 32, and the pole33 of the measurement unit 30 has been described above. The followingdescription will be made on the premise that one user uses onemeasurement unit 30. However, one measurement apparatus 31 may measuremovements of the bodies of a plurality of users. That is, a plurality ofusers may take exercise within the measurement range of one measurementapparatus 31.

The terminal apparatus 10 and the server 20 are connected to andcommunicate with each other via a network 80. In this embodiment, eachapparatus (the terminal apparatus, the server or the like) can also beregarded as an information processing apparatus. That is, each unit ofapparatus can be regarded as one “information processing apparatus”, andthe measurement system 1 can be formed as a set of a plurality ofapparatuses. A plurality of required functions of the measurement system1 according to this embodiment can be appropriately assigned to one ormore pieces of hardware in view of the processing capability of eachpiece of hardware and/or the required specifications of the measurementsystem 1.

The terminal apparatus 10 is an apparatus operated by a user. Theterminal apparatus 10 may be a desktop personal computer (PC) or alaptop PC, for example. Alternatively, the terminal apparatus 10 may beimplemented as a portable terminal or the like that supports a mobilecommunication system, such as a smartphone or a tablet. As shown as theterminal apparatus 10 in FIG. 1, the terminal apparatus 10 includes acommunication interface (IF) 12, an input apparatus 13, an outputapparatus 14, a memory 15, a storage unit 16, and a processor 19. Theserver 20 includes a communication IF 22, an input/output IF 23, amemory 25, a storage 26, and a processor 29.

The terminal apparatus 10 is communicatively connected to the server 20via the network 80. The terminal apparatus 10 supports a communicationstandard, such as 5G or Long Term Evolution (LTE), and a wireless localarea network (LAN) standard, such as Institute of Electrical andElectronics Engineers (IEEE) 802.11, and is connected to the network 80by communicating with communication equipment.

The communication IF 12 is an interface for inputting and outputting asignal that allows the terminal apparatus 10 to communicate with anexternal apparatus. The input apparatus 13 is an input apparatus (suchas a touch panel, a touch pad, a pointing device such as a mouse, or akeyboard) that receives an input operation by the user. The outputapparatus 14 is an output apparatus (such as a display or a speaker)that presents information to the user. The memory 15 temporarily storesa program and data or the like processed by a program or the like, andis a volatile memory, such as a dynamic random access memory (DRAM). Thestorage unit 16 is a storage apparatus for retaining data, and is aflash memory or a hard disc drive (HDD), for example. The processor 19is hardware for executing an instruction set described in a program, andis formed by an arithmetic apparatus, a register, and a peripheralcircuit, for example.

As shown in the drawing, the terminal apparatus 10 connects to themeasurement unit 30 by wire or wirelessly. By the measurement unit 30measuring the movement of the body of the user, the flexibility and thejoint range of motion of the body of the user can be evaluated.

The server 20 manages user information. As the user information, theserver 20 manages a parameter (such as height) or the like that isreferred to when measuring the movement of the body of a user.

The server 20 performs processing for promoting relations between users.Specifically, the server 20 performs a match-making between users tohelp a plurality of users take exercise for health and measure themovements of their bodies. The server 20 is also responsible for messageexchange between users, for example.

The server 20 may manage virtual loyalty points of a user. For example,suppose that a business company has a loyalty point system, and theemployees can use their loyalty points as virtual currency in thecompany. As with the virtual currency, the loyalty point can be used fora purchase in the company (such as payment at vending machines or thelike in the company) or transferred to another employee. For example,the server 20 may award the corresponding points to a user when the userperforms measurements with the measurement system 1, may award thecorresponding points to a user when the user performs measurements withthe measurement system 1 together with another user (such as when theuser performs a measurement at the same place, or at the same place atthe same time, as another user), or may award the corresponding pointsto a user when the evaluation result of a joint range of motion exceedsa predetermined value.

The communication IF 22 is an interface for inputting and outputting asignal that allows the server 20 to communicate with an externalapparatus. The input/output IF 23 serves as an interface to an inputapparatus that receives an input operation by the user or an outputapparatus that presents information to the user. The memory 25temporarily stores a program and data or the like processed by a programor the like, and is a volatile memory, such as a dynamic random accessmemory (DRAM). The storage 26 is a storage apparatus for retaining data,and is a flash memory or a hard disc drive (HDD), for example. Theprocessor 29 is hardware for executing an instruction set described in aprogram, and is formed by an arithmetic apparatus, a register, and aperipheral circuit, for example.

<1.1 Configuration of Terminal Apparatus 10>

FIG. 2 is a block diagram showing the terminal apparatus 10 forming themeasurement system 1 according to the first embodiment. As shown in FIG.2, the terminal apparatus 10 includes a plurality of antennas (anantenna 111 and an antenna 112), wireless communication units (a firstwireless communication unit 121 and a second wireless communication unit122) associated with the antennas, an operation receiving unit 130(which includes a touch sensitive device 131, and may further includevarious pointing devices, such as a touch pad, or include a keyboard ora gesture detection apparatus), a display 132, an audio processing unit140, a microphone 141, a speaker 142, a position information sensor 150,a camera 160, a storage unit 180, and a control unit 190. The terminalapparatus 10 further has a function or component that is notspecifically shown in FIG. 2 (such as a battery for holding electricpower or a power supply circuit that controls the supply of electricpower from the battery to each circuit). As shown in FIG. 2, the blocksincluded in the terminal apparatus 10 are electrically connected to eachother by a bus or the like.

The antenna 111 radiates a signal emitted by the terminal apparatus 10in the form of a radio wave. The antenna 111 also receives a radio wavein the air and provides the received signal to the first wirelesscommunication unit 121.

The antenna 112 radiates a signal emitted by the terminal apparatus 10in the form of a radio wave. The antenna 112 also receives a radio wavein the air and provides the received signal to the second wirelesscommunication unit 122.

The first wireless communication unit 121 performs modulation anddemodulation processing for transmitting and receiving a signal via theantenna 111 to allow the terminal apparatus 10 to communicate with otherwireless equipment. The second wireless communication unit 122 performsmodulation and demodulation processing for transmitting and receiving asignal via the antenna 112 to allow the terminal apparatus 10 tocommunicate with other wireless equipment. The first wirelesscommunication unit 121 and the second wireless communication unit 122are communication modules including a tuner, a received signal strengthindicator (RSSI) calculation circuit, a cyclic redundancy check (CRC)calculation circuit, and a high-frequency circuit, for example. Thefirst wireless communication unit 121 and the second wirelesscommunication unit 122 performs modulation and demodulation or frequencyconversion of a radio signal transmitted and received by the terminalapparatus 10, and provides the received signal to the control unit 190.

The operation receiving unit 130 has a mechanism for receiving an inputoperation by the user. Specifically, the operation receiving unit 130includes the touch sensitive device 131 or the like as the pointingdevice. The touch sensitive device 131 receives an input operation bythe user of the terminal apparatus 10. For example, the touch sensitivedevice 131 uses a capacitance-type touch panel to detect a user touchpoint on the touch panel. The touch sensitive device 131 outputs asignal indicating the user touch point detected by the touch panel tothe control unit 190 as an input operation. The terminal apparatus 10may receive an input operation by the user by communicating with themeasurement apparatus 31 and receiving the measurement result from themeasurement apparatus 31. For example, when the measurement apparatus 31obtains point cloud data, the terminal apparatus 10 receives a gestureof the user as an input by detecting the body of the user and jointsforming the body of the user based on the point cloud data andrecognizing the gesture based on the detection result. The gesture ofthe user that can be input may be a movement of the body of the user,such as a movement of a hand or a movement of the head. For example, theoperation receiving unit 130 may detect a predetermined shape of thebody of the user (such as a “T” shape with the arms stretched outhorizontally) or a predetermined shape of a hand of the user.

The display 132 displays data, such as an image, a video, or a text,under the control of the control unit 190. The display 132 isimplemented by a liquid crystal display (LCD) or an organicelectro-luminescence (EL) display, for example. The output apparatus(the display 132 and the speaker 142) may be implemented as a deviceseparate from the terminal apparatus 10.

The audio processing unit 140 performs modulation and demodulation of anaudio signal. The audio processing unit 140 modulates a signal providedfrom the microphone 141 and provides the modulated signal to the controlunit 190. The audio processing unit 140 also provides the audio signalto the speaker 142. The audio processing unit 140 is implemented by aprocessor for audio processing, for example. The microphone 141 receivesan audio input and provides an audio signal corresponding to the audioinput to the audio processing unit 140. The speaker 142 converts theaudio signal provided by the audio processing unit 140 into a sound andoutputs the sound to the outside of the terminal apparatus 10.

The position information sensor 150 is a sensor that detects theposition of the terminal apparatus 10, and is a global positioningsystem (GPS) module, for example. The GPS module is a receivingapparatus used in the satellite positioning system. In the satellitepositioning system, the GPS module receives signals from at least threeor four satellites, and detects the current position of the terminalapparatus 10 in which the GPS module is incorporated based on thereceived signals. For example, when the measurement system 1 records theplace where the user has performed measurements, the detection result ofthe position information sensor 150 (such as latitude and longitudeinformation, information on a facility identified from the latitude andlongitude information and map information, or information on a businessentity associated with the facility) may be associated with themeasurement result and retained in the server 20. That is, the terminalapparatus 10 may determine the place where the user has performedmeasurements based on the detection result of the position informationsensor 150, rather than relying on the operation of the user inputtingthe place.

The camera 160 is a device that receives light at a light-receivingelement and outputs the received light as a taken image. The camera 160is a depth camera capable of detecting the distance from the camera 160to the subject. In this embodiment, the measurement apparatus 31 may beimplemented by the camera 160 of the terminal apparatus 10. That is, theterminal apparatus 10 and the measurement apparatus 31 does not have tobe implemented as separate apparatuses, and the terminal apparatus 10may have the functionality of the measurement apparatus 31.

The storage unit 180 is formed by a flash memory, for example, andstores data and a program to be used by the terminal apparatus 10. In asituation, the storage unit 180 stores user information 181, measurementitem information 182, measurement history information 183, a friend list184, and training item information 185. Although these will be describedin detail later, these are summarized as follows. The terminal apparatus10 may obtain these pieces of information from various databases of theserver 20 described later.

The user information 181 is information on a user of the measurementsystem 1. The user information includes information for identifying theuser, the name of the user, an affiliation of the user, or informationon the body of the user (such as height, weight, or age).

The measurement item information 182 is information on measurement itemsin the case of performing ROM check. The measurement item information182 includes information on an audio guide to be output to the user fora measurement item.

The measurement history information 183 indicates a history ofmeasurement results of a movement of the body of the user.

The friend list 184 indicates information on a user registered as afriend by the user. Users who have registered each other as friends maybe able to transmit and receive messages.

The training item information 185 includes information on a trainingitem in the case of providing a muscle training regimen to the user. Forexample, the training item information 185 includes the name of thetraining and information on video and audio serving as a guide for theuser to take exercise. The terminal apparatus 10 measures, with themeasurement apparatus 31, the user taking exercise according to thevideo and audio serving as a guide, and sequentially displays themeasurement result on the display 132. The terminal apparatus 10 maydisplay for comparison the ideal movement of the body and themeasurement result (movements of joints) of the user taking exercise onthe display 132 so as to facilitate the user to move his or her body inan ideal way.

The control unit 190 controls an operation of the terminal apparatus 10by reading in a program stored in the storage unit 180 and executing aninstruction included in the program. The control unit 190 is anapplication processor, for example. The control unit 190 operatesaccording to the program to serve functions as an input operationreceiving unit 191, a transmission/reception unit 192, a data processingunit 193, and a notice control unit 194.

The input operation receiving unit 191 performs processing of receivingan input operation of the user on the input apparatus such as the touchsensitive device 131. The input operation receiving unit 191 determinesthe type of the operation by the user, such as whether the operation isa flick operation, a tap operation or a drag (swipe) operation, based oninformation on the coordinates of the point where the user has touchedthe touch sensitive device 131 with a finger or the like.

The transmission/reception unit 192 performs processing for the terminalapparatus 10 to transmit or receive data to or from an externalapparatus, such as the server 20 or the measurement unit 30, accordingto a communication protocol.

The data processing unit 193 performs processing of performing acalculation on input data received by the terminal apparatus 10according to a program and outputting the calculation result to a memoryor the like.

The notice control unit 194 performs processing of making the display132 display an image or processing of making the speaker 142 outputaudio. For example, the notice control unit 194 outputs an audio guidewhen performing ROM check, and presents an audio guide and video showingan exemplary movement of the body when providing a training regimen forexercise.

<1.2 Functional Configuration of Server 20>

FIG. 3 is a diagram showing a functional configuration of the server 20.As shown in FIG. 3, the server 20 serves functions as a communicationunit 201, a storage unit 202, and a control unit 203.

The communication unit 201 performs processing for the server 20 tocommunicate with an external apparatus, such as the terminal apparatus10.

The storage unit 202 stores data and a program to be used by the server20. The storage unit 202 stores a user information database 281,measurement item information 282, a measurement history database 283,and a friend list 284, for example.

The user information database 281 indicates information on the user whoperforms measurements with the measurement system 1. The userinformation database 281 will be described in detail later.

The measurement item information 282 indicates information on each of aplurality of measurement items. The measurement item information 282will be described in detail later.

The measurement history database 283 indicates a history of measurementsperformed by the user. The measurement history database 283 will bedescribed in detail later.

The friend list 284 is a database for retaining information on a userregistered as a friend by the user.

Training item information 285 includes information on a training item inthe case of providing a training regimen for exercise to the user. Thetraining regimen includes information on a plurality of training items,depending on the load on the body of the user or the part of the body tobe worked on.

The control unit 203 serves functions of various modules by theprocessor of the server 20 performing processing according to a program.

An operation detail obtaining module 2041 obtains details of theoperation by the user. For example, the operation detail obtainingmodule 2041 receives an operation for the user to obtain a pastmeasurement result from the measurement unit 30 and a past evaluationresult of the movement of the body of the user, an operation fortransmitting and receiving a message to or from another user, or anoperation for applying for participation in a measurement with themeasurement unit 30, for example.

A reception control module 2042 controls processing of the server 20receiving a signal from an external apparatus according to acommunication protocol.

A transmission control module 2043 controls processing of the server 20transmitting a signal to an external apparatus according to acommunication protocol.

A match-making module 2044 makes a match between users. For example, thematch-making module 2044 may detect that a plurality of users hasperformed measurements with the measurement system 1 at the samemeasurement site, make a match between the users, and prompt the usersto register each other as friends. The match-making module 2044 mayextract users having similar histories of measurements with themeasurement system 1 and make a match between the users, or extractusers having similar results of the ROM check with the measurement unit30 and make a match between the users. By making a match between aplurality of users and prompting the users to register each other asfriends, the plurality of users can be prompted to have a training forgetting a good posture together. Therefore, the measurement system 1 canassist the users to continue measurements.

A measurement assistance module 2045 assists the progress of theprocessing of the measurement unit 30 measuring the movement of the bodyof the user. The measurement assistance module 2045 reads measurementitems indicated by the measurement item information 282 and sequentiallyoutputs an audio guide or the like for the measurement of themeasurement items to the user. For example, the server 20 may transmitmeasurement items to the terminal apparatus 10 to make the terminalapparatus 10 sequentially measure the movement of the body of the useraccording to the measurement items.

A training assistance module 2046 assists the progress of the processingof providing a training regimen through video and audio to the user. Thetraining assistance module 2046 reads training items indicated by thetraining item information 285 and sequentially outputs an audio guide, avideo guide or the like for training according to the training items.

<2. Data Structure>

FIG. 4 is a diagram showing data structures of the user informationdatabase 281, the measurement item information 282, the measurementhistory database 283 and the friend list 284 stored in the server 20.

As shown in FIG. 4, each record of the user information database 281includes an item “user identification information (user ID)”, an item“user name”, an item “sex”, an item “height”, an item “weight”, an item“affiliation”, an item “evaluation value”, and an item “company loyaltypoint”, for example.

The item “user identification information (user ID)” is information foridentifying the user.

The item “user name” is the name set by the user.

The item “sex” indicates the sex of the user. For example, the user mayselect his or her sex when registering user information to start usingthe measurement system 1.

The item “height” indicates the height of the user, which is informationto be referred to when measuring the movement of the body of the user.

The item “weight” is information on the weight of the user. If theweight information is not referred to when measuring the movement of thebody of the user, for example, the item “weight” may contain no data.

The item “affiliation” is information on the affiliation of the user whobelongs to an organization, for example. The item “affiliation” includesinformation on a company name, an organization name, or a businessoffice name, for example. The item “affiliation” may include informationon the address of the office where the user works.

The item “evaluation value” includes information on an evaluation valuebased on the measurement result of the ROM check of the user with themeasurement system 1 or an evaluation value based on the measurementresult of the posture of the user walking. If the terminal apparatus 10classifies the results of ROM check of the user based on the measurementresult with the measurement unit 30, the evaluation value includesinformation on the result of the classification.

The item “company loyalty point” indicates the amount of points of theuser when the business company to which the user belongs has a virtualloyalty point system.

Each record of the measurement item information 282 includes an item“measurement item number (measurement item No.)”, an item “measuredpart”, an item “audio guide”, an item “foot positions on mat”, and anitem “body part to be measured”.

The item “measurement item number (measurement item No.)” is informationfor identifying the measurement item.

The item “measured part” indicates the part of the body to be measured.For example, parts for which the joint range of motion is evaluated areas follows.

(i) Measurement of Upper Limb

-   -   Scapula: the movements include flexion, extension, elevation,        and depression.    -   Shoulder: the movements include flexion, extension, abduction,        adduction, external rotation, and internal rotation.    -   Elbow: the movements include flexion and extension.    -   Forearm: the movements include pronation and supination.    -   Wrist: the movements include flexion, extension, radial flexion,        and ulnar flexion.

(ii) Measurement of Fingers

-   -   Thumb: the movements include radial abduction, ulnar adduction,        palmar abduction, palmar adduction, flexion, and extension.    -   Fingers: the movements include flexion, extension, abduction,        and adduction.

(iii) Measurement of Lower Limb

-   -   Hip: the movements include flexion, extension, abduction,        adduction, external rotation, and internal rotation.    -   Knee: the movements include flexion and extension.    -   Ankle: the movements include flexion and extension.    -   Foot: the movements include eversion, inversion, abduction, and        adduction.    -   Big toe: the movements include flexion and extension.    -   Toes: the movements include flexion and extension.

(iv) Measurement of Trunk

-   -   Cervical spines: the movements include flexion, extension,        rotations (left rotation and right rotation), and lateral        flexions (left flexion and right flexion).    -   Thoracic and lumbar spines: the movements include flexion,        extension, rotations (left rotation and right rotation), and        lateral flexions (left flexion and right flexion).

The item “audio guide” indicates details of the audio presented to theuser when performing the measurement indicated by the measurement item.The audio guide includes audio for the case where the user does not puthis or her feet at prescribed positions on the mat 32 (such as “put yourfeet at the prescribed positions on the mat”), audio for prompting theuser to move his or her body after the terminal apparatus 10 detectsthat the user has put his or her feet at the prescribed positions on themat 32 for measurement (such as “twist your body with your feet kept atthe prescribed positions on the mat”), and audio for the case wheremeasurement data on the movement of the body of the user has beenobtained based on the measurement result from the measurement apparatus31 (such as “the measurement is normally made”).

The item “foot positions on mat” includes information on where the usershould put his or her feet on the mat 32 for measurement (the positionsat which the user should put his or her feet on the mat 32). Forexample, the terminal apparatus 10 detects whether the user has put hisor her feet at the prescribed positions on the mat or not based on theinformation of the item “foot positions on mat”. Based on the detection,the terminal apparatus 10 outputs to the user audio indicating detailsof the movement of the body to be measured.

The item “body part to be measured” indicates the part of the body ofthe user that is to be measured for the measurement item.

Each record of the measurement history database 283 includes an item“measurement number (measurement No.)”, an item “date and time”, an item“user identification information (ID)”, an item “place of measurement”,an item “measurement data”, and an item “evaluation result”. The server20 receives the measurement result of the measurement of the movement ofthe body of the user from the terminal apparatus 10. The server 20updates the records in the measurement history database 283 based on theinformation received from the terminal apparatus 10.

The item “measurement number (measurement No.)” indicates a measurementresult of measurement of the movement of the body of the user with themeasurement unit 30.

The item “data and time” indicates the timing when the user performedmeasurements with the measurement unit 30.

The item “user identification information (ID)” is information foridentifying the user who has measured the movement of the body with themeasurement unit 30.

The item “place of measurement” indicates the place where the userperformed measurements with the measurement unit 30. When the server 20receives the measurement result from the terminal apparatus 10, theserver 20 may refer to information on the user associated with themeasurement result, and determine the place where the user performed themeasurement based on the information (user information database 281)associated with the user. For example, based on the information on theaffiliation of the user indicated in the user information database 281,the server 20 may regard the information on the place of the affiliationas the place where the user performed the measurement with themeasurement unit 30. Alternatively, when the terminal apparatus 10obtains position information with the position information sensor 150,for example, the place of the terminal apparatus 10 may be regarded asthe place where the user performed the measurement with the measurementunit 30.

The item “measurement data” indicates the measurement result of themovement of the body of the user measured with the measurement unit 30.For example, when the movement of the body of the user is measured astime-series point cloud data, the time-series point cloud data may beregarded as the measurement result for the item “measurement data”.Alternatively, for the body of the user measured by the measurementapparatus 31, the terminal apparatus 10 may determine the positions ofjoints of the user based on the measurement result, and the server 20may retain the movements of the joints as the measurement result of themovement of the body of the user. For example, the terminal apparatus 10may measure temporal movements of joints of the user by detectingcertain parts of the body, such as a shoulder or an elbow, based on ataken image of the body of the user.

The item “evaluation result” is an evaluation result that indicates thelevel of the movement of the body of the user based on the measurementresult indicated by the item “measurement data”. When the terminalapparatus 10 retains time-series data on the movement of a joint of theuser as the item “measurement data”, for example, the terminal apparatus10 evaluates the movement of the body of the user by comparing themovement with the reference data on the movement of the body. Forexample, if the terminal apparatus 10 retains data on ranking of qualityof the movement of the body of the user as reference data, the terminalapparatus 10 can determine the reference data on the movement closest tothe measurement result of the movement of the body of the user.

Each record of the friend list 284 includes an item “user 1), an item“user 2”, an item “friend registration date”, an item “history ofmeasurement at same timing”, and an item “history of message exchange”,for example.

The item “user 1” indicates one of the users who have registered eachother as friends.

The item “user 2” indicates the other of the users who have registeredeach other as friends.

The item “friend registration date” indicates the date and time when theusers registered each other as friends.

The item “history of measurement at same timing” indicates a history ofmeasurements performed by the users registered as friends at the sameplace and/or at the same timing. For example, when the measurement unit30 is disposed in the office space or the like of a business office, aplurality of users can be regarded as having performed measurements atthe same timing if the users have performed measurements in a certaintime period. The item “history of measurement at same timing” mayinclude information on the places where a plurality of users performedmeasurements. For example, by referring to the friend list 284, theserver 20 can determine the frequency at which a plurality of usersperforms measurements of the movement of the body with the measurementunit 30 and conducts a training according to a training regimen.

The item “history of message exchange” indicates a history of messagestransmitted and received between the users registered as friends.

<3. Configuration of Measurement Unit 30>

FIG. 5 shows a situation where users measure the movement of the body.FIG. 5 shows an example where a plurality of measurement units 30 isinstalled, and a plurality of users performs measurements. In theexample shown, one terminal apparatus 10 is connected to the pluralityof measurement units 30.

The measurement apparatus 31A measures the movement of the body of afirst user User_A in a detection range 38A, in which the measurementapparatus 31A can sense the body of the user. In the detection range38A, the mat 32A is arranged in a predetermined positional relationshipto the measurement apparatus 31A. Here, the position of the mat 32A maybe a position at a certain distance (such as several meters or so) fromthe front of the measurement apparatus 31A, for example. The mat 32A ispreferably arranged with respect to the measurement apparatus 31A insuch a manner that the measurement apparatus 31A can measure themovement of the entire body of the first user User_A when the user is onthe mat 32A. The poles 33A are arranged on the left and right sides ofthe mat 32A. For example, when there are marks indicating prescribedpositions (which may be referred to also as home positions) for the feetof the user on the mat 32A, the pair of poles 33A may be arrangedoutside of the mat 32A with respect to the marks. The poles 33A areinstalled as a target to be grasped by the user when the user makes amovement of twisting the upper limb on the mat 32A with the feet kept atthe prescribed positions. With the poles 33A, when the user makes amovement of twisting the upper body, the user naturally reaches towardthe pole 33A, so that the form of the movement of the upper body can bestabilized to some extent even when an increased number of measurementsare performed. That is, variations of the time-series measurement resultof the movement of the body of the user can be reduced.

The measurement apparatus 31B measures the movement of the body of asecond user User_B in a detection range 38B, in which the measurementapparatus 31B can sense the body of the user. In the detection range38B, the mat 32B is arranged in a predetermined positional relationshipto the measurement apparatus 31B. Here, the position of the mat 32B maybe a position at a certain distance (such as several meters or so) fromthe front of the measurement apparatus 31B, for example. The mat 32B ispreferably arranged with respect to the measurement apparatus 31B insuch a manner that the measurement apparatus 31B can measure themovement of the entire body of the second user User_B when the user ison the mat 32B. The poles 33B are arranged on the left and right sidesof the mat 32B.

The first user User_A moves his or her body while watching video andaudio displayed on the display 132A and checking the foot positions withrespect to the marks on the mat 32A. Similarly, the second user User_Bmoves his or her body according to a video displayed on the display 132Band audio guide. In this way, the measurement apparatuses 31A and 31Bcan measure the movements of the bodies of the first user User_A and thesecond user User_B, respectively.

In the example shown, the first user User_A is about to start flexionand extension with the feet placed at the home positions on the mat 32Ain accordance with the output of the display 132A. The second userUser_B is making a movement of stepping the right foot forward inaccordance with the output of the display 132B.

FIG. 6 is a diagram showing details of the mat 32. As shown in thedrawing, a plurality of marks is arranged which serves as targets onwhich the right and left feet of the user should be placed.

For example, at target positions for the feet of the user for aplurality of measurement items, a plurality of marks for the right foot,such as marks of a first home position 41R, a second home position 42R,and a third home position 43R, are indicated.

A plurality of marks for the left foot, such as marks of a first homeposition 41L, a second home position 42L, and a third home position 43L,are also indicated.

In the following description, the home positions 41R and 41L may begenerically referred to as a home position 41 or the like. Similarly,the home positions 42R and 42L may be generically referred to as a homeposition 42, and the home positions 43R and 43L may be genericallyreferred to as a home position 43.

For each measurement item (measurement item information 282), thedistance between the left and right feet of the user is prescribed, andthe terminal apparatus 10 guides the user to perform a predeterminedbody movement. For example, information on the relevant home positionsmay be provided by the audio guide associated with the measurement ofthe measurement item.

When performing a measurement of a measurement item, the terminalapparatus 10 determines whether or not the user is using thepredetermined home positions based on the measurement result from themeasurement apparatus 31. For example, when the terminal apparatus 10discriminates the marks of the home positions on the mat 32 based on thesensing result from the measurement apparatus 31, the terminal apparatus10 may be able to determine the marks on which the feet of the user areresting based on the detection result of the ankles or the like of theuser. When performing a measurement of a measurement item, if the userputs his or her feet at different positions than the intended homepositions, the user can be notified by an audio guide, a video guide orthe like to put his or her feet at the proper positions.

As a measurement item, the terminal apparatus 10 includes that theterminal apparatus 10 measures, with the measurement unit 30, themovements of joints of the user making a movement of stepping a footforward. With this measurement item, the terminal apparatus 10 canevaluate the balance of the body of the user making a movement ofstepping a foot forward as a way of moving the body, for example.

The mat 32 includes a group of target marks 45R for the right foot for acase where a measurement item involves a movement of stepping a footforward. The mat 32 also includes a group of target marks 45L for theleft foot.

These groups of marks 45R and 45L may be generically referred to as agroup of marks 45.

The group of marks 45 includes target marks for a plurality of steppositions for users of different heights. In the example shown, marksare arranged in steps of 10 cm in the range of the height of the userfrom 140 cm to 200 cm. The mark has a shape similar to the toe part ofthe foot so that the user can easily recognize that the mark indicatesthe target position to which the user should step a foot forward. Theuser put his or her feet at the home positions 41R and 41L or marks 44Rand 44L (which may be generically referred to as a mark 44) as initialpositions, and then makes a movement of stepping his or her feet forwardone by one to the target marks, which depend on his or her height. Theterminal apparatus 10 measures, with the measurement unit 30, themovement of the body (movements of joints) of the user stepping his orher feet forward. When measuring the movement of the user stepping hisor her feet forward, the terminal apparatus 10 may guide the user byspecifying the home positions 41, 42 or 43 or the marks 44 as theinitial foot positions according to the desired distance for which theuser steps his or her feet forward.

In the example shown, the positions of the poles 33 are also shown. Forexample, the poles 33 (right and left poles) may be arranged in linewith the home positions 41, 42, and 43. For example, the user makes amovement of twisting the upper limb and tries grasping the poles 33 withhis or her feet kept at any of the home positions 41, 42 and 43.

As shown in the drawing, arrow marks may be arranged along with thegroup of marks 45R and 45L. In the example shown, the arrow marks areoverlaid on the marks for different user heights of the group of marks45R and 45L. This can remind the user to make a movement of stepping hisor her feet forward and may be able to reduce the possibility that theuser get confused about how to move his or her body for measurement whenthe user sees the mat 32.

As shown in the drawing, as for the size of the marks, since the marksare targets on which the user put his or her feet, the length of themarks in the longitudinal direction (the direction in which the usersteps his or her feet forward) in the example shown may be on the orderof a common foot size (approximately 22 cm to 28 cm). As for the shapeof the marks, the marks may have the shape of a footstep or a shapesimilar to a footstep so that the user can easily recognize that theuser should put his or her feet on the marks.

<4. Operation>

In the following, a measurement process in which the terminal apparatus10 measures the movement of the body of the user will be described.

FIG. 7 is a flowchart showing a process in which the terminal apparatus10 performs a ROM check and updates a measurement history.

In step S700, the control unit 190 of the terminal apparatus 10 displaysa log-in screen for a user who is going to perform measurements on thedisplay 132, and receives a log-in operation from the user. For example,the terminal apparatus 10 authenticates the user and obtain informationon the user by reading information from a recording medium, such as anIC card, of the user or reading a two-dimensional code displayed on acommunication terminal of the user. For example, the terminal apparatus10 communicates with the server 20 to obtain, as the information on theuser, information on the height or the like of the user (userinformation database 281), a past measurement history (measurementhistory database 283), or information on another user registered as afriend of the user (friend list 284). The terminal apparatus 10 alsoobtains information on a measurement item (measurement item information282) from the server 20, and stores the information as the measurementitem information 182.

In step S701, the terminal apparatus 10 detects that the mat 32 and thepoles 33 are installed in the measurement range of the measurementapparatus 31 based on the measurement result from the measurementapparatus 31. For example, the terminal apparatus 10 detects the mat 32and the poles 33 through image recognition of a two-dimensional imageobtained from the measurement result from the measurement apparatus 31.The terminal apparatus 10 detects that the mat 32 and the poles 33 arelocated at predetermined positions with respect to the measurementapparatus 31. That is, the terminal apparatus 10 detects that the mat 32and the poles 33 are located at positions suitable for measurement, andthe user is located at a position where the user can easily recognizethe audio and video of the guide.

In step S703, the terminal apparatus 10 refers to the measurement iteminformation 182 and performs processing of sequentially performingmeasurements of measurement items. The terminal apparatus 10 reads afirst measurement item and outputs information serving as a guide formeasurement in the form of audio or video (such as foot positions of theuser on the mat 32). The terminal apparatus 10 detects joints of theuser based on the measurement result from the measurement apparatus 31,and detects that the feet of the user are put at predetermined positions(prescribed positions for the measurement item).

In step S705, the terminal apparatus 10 outputs audio that prompts theuser to make a predetermined movement of the body. The terminalapparatus 10 detects the movement of the body of the user with themeasurement apparatus 31, thereby measuring the movement of the body ofthe user.

In step S707, the terminal apparatus 10 sequentially reads themeasurement items from the measurement item information 182, outputs anaudio guide, detects that the feet of the user are at the prescribedpositions on the mat 32, and measures the movement of the body of theuser.

In step S709, when the measurements of the measurement items arecompleted, the terminal apparatus 10 evaluates the joint range of motionof the user based on the measurement result of the movement of the bodyof the user. Specifically, the terminal apparatus 10 determines thepositions of joints of the user based on the measurement result from themeasurement apparatus 31, and evaluates the range of motion of the partof the body to be measured based on the time-series data on thepositions of the joints (the movements of the positions of the jointsthat change with time). The terminal apparatus 10 outputs the evaluationresult of the joint range of motion through the display 132 or the like,and transmits the evaluation result to the server 20.

In step S753, the server 20 receives the result of the ROM check of theuser from the terminal apparatus 10, and updates the databases (themeasurement history database 283 and the user information database 281)based on the measurement result of the user.

Next, processing performed by each apparatus when measuring the movementof the user stepping his or her feet forward as a measurement item willbe described.

FIG. 8 is a flowchart showing a process in which the terminal apparatus10 evaluates the joint range of motion by measuring the movement of theuser stepping his or her feet forward.

In step S801, the terminal apparatus 10 detects that the feet of theuser are put at the prescribed positions (the item “foot positions onmat” in the measurement item information 282) on the mat 32 based on themeasurement result from the measurement apparatus 31. If the feet of theuser are not put at the prescribed positions, the terminal apparatus 10may guide the user to put his or her feet at the prescribed positions.

In step S803, the terminal apparatus 10 outputs audio or video thatguides the user to step the right foot from the prescribed position tothe step position appropriate to the height of the user.

In step S805, the terminal apparatus 10 detects that the user hasstepped the right foot from the prescribed position to another positionon the mat based on the measurement result from the measurementapparatus 31, and records the measurement result of the movement of theuser stepping the right foot forward. When the user has stepped theright foot to a different position than the target position, theterminal apparatus 10 may guide the user to step the right foot to thetarget position again, in order to redo the measurement.

In step S807, the terminal apparatus 10 outputs audio that guides theuser to step the left foot from the prescribed position to the stepposition appropriate to the height (step length) of the user.

In step S809, the terminal apparatus 10 detects that the user hasstepped the left foot from the prescribed position to another positionon the mat based on the measurement result from the measurementapparatus 31, and records the measurement result of the movement of theuser stepping the left foot forward.

In step S811, the terminal apparatus 10 evaluates the joint range ofmotion of the lower limb based on the measurement result of the movementof the user stepping his or her feet forward on the mat 32, outputs theevaluation result through the display 132 or the like, and transmits theevaluation result to the server 20.

Next, processing performed by each apparatus when measuring the movementof the user twisting the upper part of the body will be described.

FIG. 9 is a flowchart showing a process in which the terminal apparatus10 evaluates the joint range of motion by measuring the movement of theuser twisting the upper part of the body.

In step S901, the terminal apparatus detects that the feet of the userare put at the prescribed positions on the mat based on the measurementresult from the measurement apparatus 31.

In step S903, the terminal apparatus 10 determines whether the height ofthe pole 33 is appropriate or not based on the information on the heightof the user. For example, the terminal apparatus 10 stores informationon the heights of users and information on the heights of the pole 33 inassociation with each other, and determines whether the height of thepole 33 detected from the measurement result from the measurementapparatus 31 is appropriate or not. For example, the appropriate heightof the pole 33 may be prescribed depending on the position of the handof the user standing upright. Here, the position of the hand of the userstanding upright can vary with the height of the user. That is, thetaller the user is, the higher the position of the hand of the userstanding upright can be. The terminal apparatus 10 may detect theposition (height from the ground) of the hand of the user in an uprightposture based on the measurement result from the measurement apparatus31, and determine the appropriate height of the pole 33 based on theposition of the hand. That is, the height of the pole 33 can be adjustedaccording to the measurement result of the body of the user by themeasurement apparatus 31.

For example, a first position where the pole 33 is in contact with theground and a second position where the top of the pole 33 is located maybe determined based on the measurement result from the measurementapparatus 31, and the height of the pole 33 may be determined based oninformation on the distances from the position of the measurementapparatus 31 to the first position and the second position concerningthe pole 33. Alternatively, when the pole 33 is configured to beexpandable, the pole 33 may have a mark indicating the height of thepole 33, and the terminal apparatus 10 may detect the mark indicatingthe height of the pole 33.

The terminal apparatus 10 may determine whether the position where thepole 33 is installed is appropriate or not. For example, the terminalapparatus 10 may detect a mark indicating a home position or the like onthe pole 33 and determine whether the position of the pole 33 isappropriate or not based on the positional relationship between theposition of the mark and the position where the pole 33 is in contactwith the ground.

In step S905, the terminal apparatus 10 outputs audio or video thatguides the user to twist the body rightward with the feet of the userkept at the prescribed positions on the mat and grasp the right pole(the pole 33 arranged on the right side) with the left hand and the leftpole (the pole arranged on the left side) with the right hand.

In step S907, the terminal apparatus 10 records the measurement resultof the movement of the user twisting the body rightward based on themeasurement result from the measurement apparatus 31.

In step S909, the terminal apparatus 10 outputs audio that guides theuser to twist the body leftward with the feet of the user kept at theprescribed positions on the mat and grasp the left pole with the righthand and the right pole with the left hand.

In step S911, the terminal apparatus 10 records the measurement resultof the movement of the user twisting the body leftward based on themeasurement result.

In step S913, the terminal apparatus 10 evaluates the joint ranges ofmotion of the body based on the measurement result of the movement ofthe user twisting the body, and outputs the evaluation result. In theevaluation of the joint range of motion, whether or not the user wasable to grasp the poles by twisting the upper part of the body can be anevaluation item.

If the height of the poles 33 is set to be approximately equal to theheight of the hands of the user standing upright as described above, theuser who is making a movement of twisting the upper part of the bodytries moving the arms and hands toward the poles 33 that can be seenwhile turning around. Therefore, the fluctuations of the movement of theuser twisting the upper part of the body can be reduced, and time-seriesmeasurement result data can be easily compared once the terminalapparatus 10 has accumulated the measurement results.

<5. Example Screens>

FIGS. 10A, 10B, 10C and 10D are diagrams showing examples of details ofnotifications on the terminal apparatus 10 in a situation where themovement of the body of the user is measured.

The example screen in FIG. 10A relates to a situation where the terminalapparatus 10 receives a log-in operation for measurement from the user.This screen corresponds to the processing in step S700 in FIG. 7. Forexample, when the measurement unit 30 is installed in an office, a shopspace or the like of a business company and is used by a plurality ofusers, each user is first authenticated.

As shown in FIG. 10A, the terminal apparatus 10 displays a notificationpart 132L on the display 132. The terminal apparatus 10 displays detailsof an operation for the user to log in in the notification part 132L. Inthe example shown, provided that the measurement unit 30 is installed ina business office, that the user can log in by having his or heremployee ID card scanned is shown in the notification part 132L. Theterminal apparatus 10 displays that another user is going to performmeasurements at the same timing in a notification part 132C. When aplurality of measurement units 30 is installed, and a plurality of usersperforms measurements together, such as in the case of the example shownin FIG. 5, the terminal apparatus 10 displays the current progress ofanother user who is going to perform measurements in the notificationpart 132C. In the example in FIG. 10A, the terminal apparatus 10displays, in the notification part 132C, that the other user who isgoing to perform measurements at the same timing has already completedthe log-in processing.

The example screen in FIG. 10B relates to a situation where the terminalapparatus 10 measures the movement of the user stepping a foot forward.This screen corresponds to steps S801 to S811 in FIG. 8.

As shown in FIG. 10B, the terminal apparatus 10 displays a notificationpart 132D, a notification part 132E, a guide output part 132F, and adetection result display part 132G on the display 132.

In the notification part 132D, the terminal apparatus 10 displaysinformation on the user who has logged in (that is, the user performingmeasurements). This allows the user to easily check that the userperforming measurements is not misidentified as another user.

In the notification part 132E, the terminal apparatus 10 displays ahistory of measurements of the user. Although in the example shown, thenumber of times of measurements performed in the past by the user withthe measurement unit 30 is displayed, the evaluation result of a pastmeasurement may be displayed, for example. This allows the user toeasily check that the user has continuously performed measurements, andcan motivate the user to continue measurements.

In the guide output part 132F, the terminal apparatus 10 notifies,through video and audio, the user of information on the target marks forthe step positions appropriate to the height of the user. This allowsthe user to easily imagine stepping a foot toward the target mark andeasily make the movement to be measured.

In the detection result display part 132G, the terminal apparatus 10displays a detection result from the measurement apparatus 31. Forexample, in the detection result display part 132G, the terminalapparatus 10 displays information that the feet of the user are at theprescribed positions for measurement (the feet are put at the homepositions). This allows the user to easily recognize that the user isproperly following the procedure for measurement, and performmeasurements with peace of mind that no measurement error will occur.

The example screen in FIG. 10C relates to a situation where the terminalapparatus 10 measures the movement of the user twisting the upper partof the body.

In the example screen in FIG. 10C, the terminal apparatus 10 hasdetected that the height of the poles 33 is not appropriate, and isprompting the user to expand the poles 33 to a predetermined heightthrough the display in the guide output part 132F. This screencorresponds to the processing in step S903 in FIG. 9.

The example screen in FIG. 10D relates to a situation where the terminalapparatus 10 notifies the user that the measurements are completed andthe evaluation result based on the measurement result and prompts theuser to register another user as a friend.

In the example screen in FIG. 10D, the terminal apparatus 10 displays,in the guide output part 132F, that the measurements of all of aplurality of measurement items are completed and a result of evaluationof the movements of the body of the user for the measurement items. Theterminal apparatus 10 refers to the history of past measurements of theuser and displays a result of comparison with the past measurementresult (such as that a joint range of motion has changed compared withthe previous measurement) in the guide output part 132F.

Through the display in a notification part 132M, the terminal apparatus10 prompts the user to register another user as a friend and shows areason why the other user is recommended as a friend candidate (such asthat the user has performed measurements at the same place).

BRIEF SUMMARY

(1) As described above, according to this embodiment, when performingthe ROM check, no special physical measurement instrument, such as anangle gauge, need to be applied to the body of the user, so that theuser can easily perform measurements by himself or herself by followingthe guide of the terminal apparatus 10. In addition, by performingmeasurements according to measurement items, the joint ranges of motionof the lower part and the upper part of the body of the user can beevaluated.

(2) When measuring the movement of the body of the user, the mat 32 andthe poles 33 can be used to allow the user to recognize targets of themovement of the body of the user. Therefore, the form of the movement ofthe body of the user is stabilized, so that measurement results can beaccumulated for comparison, and the user can easily check changes of themovement of the body. For example, by using the mat 32, (i) the way ofstriding, such as the step length of the user stepping a foot forward orthe range of movement, can be prescribed, and the user can be promptedto make a prescribed movement for the measurement. When continuouslymeasuring a movement of the body of the user, the comparison of themeasurement results at different times can be made easier by prescribingthe movement of the user to be measured. That is, by prescribing themovement of the body of the user, the evaluation of how the movement ofthe body of the user has changed can be made easier. In addition, byusing the mat 32, (ii) the foot positions of the user with respect tothe measurement apparatus 31 can be prescribed. The terminal apparatus10 can determine whether or not the mat 32 is arranged at apredetermined position with respect to the measurement apparatus 31based on the measurement result from the measurement apparatus 31. Whenmeasuring the movement of the body of the user on the mat 32, the footpositions of the user are prescribed by marks on the mat 32. As aresult, the movement of the body of the user can be made more likely tofall within the measurement range of the measurement apparatus 31. Forexample, even if the user makes a movement of stretching an arm straightup or to the side, the movement of the body of the user can be preventedfrom going out of the measurement range of the measurement apparatus 31if the mat 32 is arranged in advance at a predetermined position withrespect to the measurement apparatus 31. In addition, by using the mat32, (iii) when making the user make a particular movement, measurementscan be started after the user has taken a proper posture. For example,the foot positions of the user can be prescribed by home positions, andwhen making the user make a predetermined movement (such as a movementof squatting down or a movement of raising one foot), measurements canbe started from a posture (with a predetermined distance between thefeet, for example) suitable for measurement.

These allow the flexibility of joints of the whole body of the user tobe easily evaluated.

(3) Furthermore, in the first embodiment, the terminal apparatus 10 maypropose an exercise suitable for the user based on the measurementresult (such as the result of the ROM check) of the movement of the bodyof the user. For example, suppose that the terminal apparatus 10 retainsexercise regimens (the training item information 185 or 285) inassociation with the classification result of the evaluation result ofthe ROM check (the measurement history information 183 or themeasurement history database 283). That is, the terminal apparatus 10retains, in advance, information on training regimens to be proposed inassociation with the measurement result of the movement of the body ofthe user or the evaluation result thereof. The terminal apparatus 10 mayrefer to the measurement history database 283 to determine an exerciseregimen to be proposed to the user based on the result of themeasurement, such as the ROM check, of the movement of the body of theuser or the evaluation result thereof, and present the determinedexercise regimen to the user by displaying the exercise regimen on thedisplay 132, for example. For example, the joint ranges of motion of theuser can be evaluated based on the result of the ROM check. In thetraining item information 285 in the server 20, a joint range of motionrequired for the user to perform an exercise may be prescribed inadvance. For example, when the result of the ROM check shows that therange of motion of the hip joint of the user is smaller than a certainvalue (when the hit joint is stiff), a predetermined training item maybe omitted from the exercise proposed to the user (the terminalapparatus 10 may be inhibited from performing processing of displayingthe video relating to the training item). Furthermore, an exerciseregimen appropriate to the evaluation result of joints of the body ofthe user may be provided to the user based on the result of the ROMcheck.

The terminal apparatus 10 measures the movement of the body of the userperforming the training indicated by the training item, and determineswhether or not the user was able to move his or her body as prescribedfor the training. It can be expected that as a result of the usercontinuing the exercise, the muscle strength of the parts of the bodyrequired for keeping a good walking posture will be gradually improved,and the user will become able to make the movement required for theexercise. Then, the terminal apparatus 10 may determine that the userhas accomplished the exercise regimen and present another exerciseregimen to the user.

Furthermore, an exercise regimen may be proposed to the user based onthe history of exercise regimens conducted by another user who issimilar to the user in evaluation result of the measurement result ofthe movement of the body.

(4) In the above description of the first embodiment, an example hasbeen described in which the movement of the body of the user is measuredand evaluated with the measurement apparatus 31, the mat 32, and thepoles 33 installed in the real world.

As an alternative, an object corresponding to the mat 32 and objectscorresponding to the poles 33 may be arranged in a virtual space tomeasure the movement of the body of the user. For example, the userwatches the display 132 or wears a head-mounted display (HMD). Theterminal apparatus 10 displays an image of the field of view thatdepends on the settings of a virtual camera in the virtual space on thedisplay 132 or the HMD. Motion sensors are attached to parts (such asthe head, the shoulder, the elbows, the hands, the hips, the knees, andthe feet) of the body of the user, and the terminal apparatus 10 tracksthe movement of the body of the user. The user moves his or her bodywhile watching the objects arranged in the virtual space. In this way,the movement of the body of the user can be measured and evaluatedwithout the mat 32 and the poles 33 arranged in the real world.

As an alternative to the mat 32 and the poles 33 arranged in the realworld, an image corresponding to the mat 32 may be projected onto thefloor surface or the like, and images (such as stereoscopic images)corresponding to the poles 33 may be displayed in the real world. Forexample, in the real world, light may be emitted from a laser, aprojector or the like to display a virtual mat 32 and virtual poles 33that can be visually recognized by the user.

Second Embodiment

The measurement system 1 according to a second embodiment will bedescribed. In the second embodiment described below, the movement of theuser walking is measured to evaluate the walking posture and assist theuser to acquire a way of walking in a good functional posture based onthe evaluation.

FIG. 11 shows a situation where the measurement system 1 according tothe second embodiment measures the movement of the user walking.

1. Overview

The measurement apparatus 31A measures a movement of the first userUser_A walking from a predetermined position to the measurementapparatus 31A in the detection range 38A. The terminal apparatus 10Anotifies the first user User_A that the walking movement of the user isto be measured and of a way of moving the body of the user while walkingthrough the display 132A. As shown in the drawing, through thenotification, the terminal apparatus 10A prompts the user to walk byputting his or her weight on his or her front foot in the wakingmovement.

In the example in FIG. 11, coordinate axes are defined as follows. Thatis, the direction of walking with respect to the position of the firstuser User_A is a “forward direction”, the direction opposite to thedirection of walking is a “rearward direction”, and the axis defined bythe direction of walking is an “anteroposterior axis”. On the surfacewhich the feet touch during walking, the direction perpendicular to thedirection of walking with respect to the position of the first userUser_A is a “transverse axis”, the direction toward the right half ofthe body of the user is a “rightward direction”, and the directiontoward the left half of the body of the user is a “leftward direction”.The axis extending from the feet to the parietal along the trunk of thefirst user User_A is a “vertical axis”.

Through the display 132A, the terminal apparatus 10A notifies the firstuser User_A to walk from a starting point at a predetermined distancefrom the position of the measurement apparatus 31A toward themeasurement apparatus 31A. Here, the starting point where the first userUser_A starts the walking movement is a position at a distance ofseveral meters or so from the measurement apparatus 31A, for example,and may be set at a position that allows the first user User_A to repeata plurality of walking cycles (provided that one cycle includes anoperation of stepping the right foot forward and an operation ofstepping the left foot forward). The measurement apparatus 31A canmeasure a plurality of walking cycles of the first user User_A. If thefirst user User_A is not at the starting point when the measurementapparatus 31A starts measuring the walking movement of the user, theterminal apparatus 10A may prompt the first user User_A to move to thestarting point through a notification on the display 132A or the like.In response to the first user User_A being at the starting point, theterminal apparatus 10A may notify the first user that measurements ofthe walking movement will be started (or may prompt the first userUser_A to start the walking movement through a notification).

FIG. 12 shows parts of the body of the user identified by the terminalapparatus 10 based on the measurement result from the measurementapparatus 31. As feature points of the body of the user, the terminalapparatus 10 identifies joints of the four limbs, the head, theshoulders, the trunk, the pelvis, the feet, the heels, the toes, ribs,and the anterior superior iliac spines, for example. The terminalapparatus 10 identifies thoracic vertebrae (as the center of the trunk)and the center of the pelvic, for example. For example, the terminalapparatus 10 may prompt the user to make a predetermined pose (such as apose with the arms thrusted sidewards to form a T shape or a pose withthe arms stuck to the trunk), and locate the positions of joints of theuser in the measurement result from the measurement apparatus 31 basedon the measurement result of the predetermined pose from the measurementapparatus 31.

2. Data Structure

FIG. 13 is a diagram showing data structures of measurement resultdatabase 287 and evaluation result database 288 stored in the server 20according to the second embodiment.

The measurement result database 287 is a database that retains ameasurement result of a movement (such as a walking movement or amovement of maintaining the balance of the trunk) of the body of a usermeasured by the measurement apparatus 31.

each record of the measurement result database 287 includes an item“measurement data ID”, an item “date and time”, an item “user ID”, anitem “elapsed time from start of measurement”, and an item “measurementdata”.

The item “measurement data ID” indicates information for identifyingmeasurement data obtained by the measurement apparatus 31. In the secondembodiment, the terminal apparatus 10 starts and ends measurements ofthe movement of the body of the user (such as measurements of thewalking cycle), and retains information for identifying each piece ofmeasurement data in association with the measurement data in the server20.

The item “date and time” indicates the date and time at which themeasurement apparatus 31 measured the movement of the body of the user.

The item “user ID” is information for identifying the user the movementof the body of whom is measured by the measurement apparatus 31.

The item “elapsed time from start of measurement” indicates the elapsedtime since the measurement of the movement of the body of the user bythe measurement apparatus 31 is started.

The item “measurement data” indicates the position of each part (such asthe head, the shoulders, and so on) of the body of the user describedabove with reference to FIG. 12 at the timing indicated by the item“elapsed time from start of measurement”. For example, in the item“measurement data”, a measurement result of the distance from theposition of the measurement apparatus 31 to each part of the body of theuser in a coordinate system with respect to the position of themeasurement apparatus 31 may be retained.

The evaluation result database 288 is a database that retains anevaluation result of the movement (such as the walking movement) of thebody of the user based on the measurement result (measurement resultdatabase 287) from the measurement apparatus 31.

Each record of the evaluation result database 288 includes an item“evaluation result ID”, an item “measurement data ID”, an item “subjectof measurement”, and an item “evaluation result”.

The item “evaluation result ID” is information for identifying anevaluation result of the measurement result of the movement of the bodyof the user from the measurement apparatus 31.

The item “measurement data ID” is information for identifying ameasurement result (measurement result database 287) from themeasurement apparatus 31 that is to be evaluated.

The item “subject of measurement” indicates a detail of the movement ofthe body of the user that is to be measured. For example, the subject ofmeasurement may be the posture of the user in the walking movement orthe balance of the trunk of the user.

The item “evaluation result” indicates an evaluation result of themovement of the body of the user based on the measurement result fromthe measurement apparatus 31. For example, when evaluating the postureof the user in the walking movement, the subject of evaluation may be “astepping forward of a foot of the user”, “whether the user is properlyshifting his or her weight onto the front foot”, or “whether the userhas completely put his or her weight on the front foot”, for example.Details will be described later. When evaluating the balance of thetrunk of the user, the subject of evaluation may be “whether thepositions of the shoulders, the hips, or the feet are misaligned”, forexample. Details will be described later.

3. Operation

In the following, a process in which the terminal apparatus 10 measuresthe movement of the body of the user and performs an evaluation based onthe measurement result in the second embodiment will be described.

FIG. 14 is a flowchart showing a process in which the terminal apparatus10 measures and evaluates the walking movement of the user in the secondembodiment.

In step S1403, the terminal apparatus 10 detects that the user is at apredetermined position in the measurement range of the measurementapparatus 31 based on the measurement result from the measurementapparatus 31.

In step S1405, the terminal apparatus 10 outputs a guide (audio orvideo) that prompts the user to start the walking movement. The terminalapparatus 10 detects joints of the user based on the measurement resultfrom the measurement apparatus 31, and records the positions of thejoints of the user in the walking movement.

In step S1407, the terminal apparatus 10 classifies a cycle of thewalking movement of the user into a plurality of phases based on themeasurement result from the measurement apparatus 31. The terminalapparatus 10 determines an evaluation value for the walking movementaccording to an evaluation item in each phase of the walking movement ofthe user.

In step S1409, the terminal apparatus 10 outputs the measurement resultfrom the measurement apparatus 31 and the evaluation result of thewalking movement through the display 132 or the like along with an idealmodel of the walking posture. The terminal apparatus 10 transmits themeasurement result and the evaluation result to the server 20.

In step S1453, the server 20 updates the measurement result database 287and the evaluation result database 288 based on the measurement resultand the evaluation result of the walking movement of the user.

4. Example Outputs of Measurement Result and Evaluation Result

FIGS. 15A, 15B, and 15C are diagrams showing example outputs in the casewhere the terminal apparatus 10 outputs a measurement result of thewalking movement of the user and an evaluation result of the walkingmovement of the user based on the measurement result in the secondembodiment. In the examples shown, the terminal apparatus 10 displaysthe measurement result and the evaluation result on the display 132.However, the terminal apparatus 10 may output the measurement result andthe evaluation result by printing the results on a paper medium or thelike with a printer or the like.

FIG. 15A relates to a situation where the terminal apparatus 10 promptsthe user to walk a predetermined number of steps (such as several metersor so) toward the measurement apparatus 31 in order to measure thewalking movement of the user. This situation corresponds to theprocessing in steps S1403 and S1405.

When the user starts the walking movement, the terminal apparatus 10displays a “notification to prompt the user to walk by putting his orher weight on the front foot” in a display part 132P1 on the display132. In display parts 132P2 and 132P3, the terminal apparatus 10displays images indicating the located positions of parts of the body ofthe user (the parts of the body illustrated in FIG. 12) as measurementresults from the measurement apparatus 31. The terminal apparatus 10displays the measurement result of the body of the user in a morevisually intuitive manner for the user by connecting the parts of thebody of the user by lines. In the example in FIG. 15A, the parts of theright half of the body of the user are connected by dotted lines, andthe parts of the left half are connected by solid lines. This allows theuser to easily discriminate between the movements of the right half ofthe body of the user and the movements of the left half of the body ofthe user in the measurement result from the measurement apparatus 31.

In the display part 132P2, the terminal apparatus 10 displays a resultof location of the parts of the user viewed from the front of the user(along the anteroposterior axis shown in FIG. 11) based on themeasurement result from the measurement apparatus 31 as required.

In the display part 132P3, the terminal apparatus 10 displays a resultof location of the parts of the user viewed from the side of the user(along the transverse axis shown in FIG. 11) based on the measurementresult from the measurement apparatus 31 as required. That is, theterminal apparatus 10 displays measurement results from the measurementapparatus 31 in the display parts 132P2 and 132P3 in real time.Therefore, the user can obtain feedback of the measurement result of themovement of the body of himself or herself in real time.

The terminal apparatus 10 notifies the user to perform the walkingmovement so as to come closer to the measurement apparatus 31. Here, theterminal apparatus 10 may display the measurement result in the displayparts 132P2 and 132P3 as if the movement of the body of the user ismeasured at a position at a predetermined distance from the body of theuser. For example, the terminal apparatus 10 creates a three-dimensionalvirtual space and arranges objects corresponding to the positions of theparts of the body of the user in the three-dimensional virtual spacebased on the measurement result from the measurement apparatus 31 (thatis, the distances from the position of the measurement apparatus 31 tothe positions of the parts of the first user User_A). For example, theterminal apparatus 10 determines the coordinates corresponding to themeasurement apparatus 31 in the three-dimensional virtual space, andarranges the object corresponding to each part at coordinatescorresponding to the position of the part with respect to thecoordinates corresponding to the measurement apparatus 31 based on themeasurement result from the measurement apparatus 31. The terminalapparatus 10 retains information on the positions of the objectscorresponding to the parts in the three-dimensional virtual space inassociation with the elapsed time during the measurement by themeasurement apparatus 31. This means that the walking movement of theuser can be reproduced as trajectories of the parts of the body of theuser by reproducing the trajectories of the movements of the objectscorresponding to the positions of the parts of the body of the user inthe three-dimensional virtual space. The terminal apparatus 10 may set avirtual camera at coordinates in a certain positional relationship tothe group of objects corresponding to the positions of the parts of theuser (such as a position at a certain distance from a “center betweenthe shoulders” along the anteroposterior axis or a position at a certaindistance from the thoracic and lumbar spines along the transverse axis),and move the virtual camera in association with the movements of theobjects corresponding to the positions of the parts.

In a display part 132P4, the terminal apparatus 10 notifies ofinformation for properly performing the measurement by the measurementapparatus 31. This corresponds to step S1403 in FIG. 14. For example,when the measurement apparatus 31 measures the walking movement of theuser, the terminal apparatus 10 determines whether the position wherethe user will start walking is at a predetermined position or not andmakes a notification to prompt the user to start walking from thepredetermined position. As shown in the drawing, when the position wherethe user will start walking is too close to the measurement apparatus31, the terminal apparatus 10 makes a notification to prompt the user tostart the walking movement from a more rearward position.

FIG. 15B relates to situation where the terminal apparatus 10 outputs ameasurement result and an evaluation result of a phase of the pluralityof phases of the walking movement of the user. This corresponds to theprocessing in step S1409.

In FIG. 15B and FIG. 15C, the measurement result of the body of the useris displayed with respect to the vertical axis. As a result, the way ofrotation of the body of the user with respect to the vertical axis canbe presented to the user.

On the display 132, the terminal apparatus 10 outputs evaluation resultsof the balance of the rotation of the shoulders and the balance of therotation of the hips in the walking movement as an evaluation result ofthe appropriateness of the walking posture of the user. The terminalapparatus 10 divides the walking movement into a plurality of phases,and outputs an evaluation result of the balance of the rotation of theshoulders, the balance of the rotation of the hips or the like for eachphase. By presenting measurement results and evaluation results for aplurality of phases of the walking movement to the user in this way, asuggestion that improves the appropriateness of the walking movement ofthe user can be provided.

In a display part 132P5, the terminal apparatus 10 displays an angle ofrotation of the shoulders for each of the counterclockwise rotation andthe clockwise rotation in the cycle of the walking movement (providedthat one cycle includes an operation of stepping the right foot forwardand an operation of stepping the left foot forward).

In a display part 132P6, the terminal apparatus 10 displays a “lineconnecting the shoulders” defined by the positions of the right shoulderand the left shoulder, a “line connecting the hip joints” defined by thepositions of the hip joints, a “central position between the shoulders”that is the center between the position of the right shoulder and theposition of the left shoulder, and a “central position of the hips”defined by the positions of the hip joints (such as the positions of theright and left halves of the pelvis). In one cycle of the walkingmovement, the terminal apparatus 10 evaluates the degree of thecounterclockwise rotation of the shoulders (to what extent the shouldershave rotated) about the vertical axis based on the position of the leftshoulder (or the positions of both the shoulders) and the positions ofthe hip joints at the time when the user has stepped the left footforward. For example, the terminal apparatus 10 draws the “lineconnecting the shoulders” based on the amount of the counterclockwiserotation of the shoulders with respect to the position of the hips inone cycle. In the display part 132P6, the terminal apparatus 10 displaysa score (“4.8 points”) as the evaluation result.

In a display part 132P7, the terminal apparatus 10 displays the “lineconnecting the shoulders”, the “line connecting the hip joints”, the“central position between the shoulders”, and the “central position ofthe hips”. In one cycle of the walking movement, the terminal apparatus10 evaluates the degree of the clockwise rotation of the shoulders aboutthe vertical axis based on the position of the right shoulder (or thepositions of both the shoulders) and the positions of the hip joints atthe time when the user has stepped the right foot forward. The terminalapparatus 10 draws the “line connecting the shoulders” based on theamount of the clockwise rotation of the shoulders with respect to theposition of the hips in one cycle. In the display part 132P7, theterminal apparatus 10 displays a score (“4.5 points”) as the evaluationresult, and the score is lower than the score for the counterclockwiserotation.

In the display parts 132P6 and 132P7, the terminal apparatus 10 displaysthe “central position between the shoulders” and the “central positionof the hips” in the movement of stepping the left foot forward and themovement of stepping the right foot forward in one cycle of the walkingmovement. This allows the user to recognize the way in which the upperpart of the body is inclined with respect to the position of the hips inthe walking movement. For example, the user can recognize the degree offluctuation of the upper part of the body with respect to the transverseaxis or the degree of fluctuation of the upper part of the body withrespect to the anteroposterior axis. In the example shown, as shown inthe display part 132P6, in the counterclockwise rotation, the degree ofoverlapping of the “central position between the shoulders” and the“central position of the hips” is high, and the upper part of the bodydoes not fluctuate. To the contrary, as shown in the display part 132P7,in the clockwise rotation, the degree of overlapping of the “centralposition between the shoulders” and the “central position of the hips”is relatively low, and the upper part of the body more significantlyfluctuates in the lateral direction than in the counterclockwiserotation. Therefore, the score as the evaluation result is lower in theclockwise rotation than in the counterclockwise rotation.

FIG. 15C relates to a situation where the terminal apparatus 10 outputsa measurement result and an evaluation result at the time when the userhas put the front foot on the ground in the walking movement, as a phaseof the walking movement of the user. This corresponds to step S1409 inFIG. 14.

In a display part 132P8, the terminal apparatus 10 displays ameasurement result and an evaluation result of the angle of rotation ofthe hips at the time when the user has put the front foot on the groundin the walking movement.

In a display part 132P9, the terminal apparatus 10 displays the “lineconnecting the hip joints” and the like at the time when the user hasput the left foot as a front foot on the ground. As shown in thedrawing, the “line connecting the hip joints” is oblique, which showsthat the user has properly rotated the hips at the time when the userhas put the left foot on the ground.

In a display part 132P10, the terminal apparatus 10 displays the “lineconnecting the hip joints” and the like at the time when the user hasput the right foot as a front foot on the ground. As shown in thedrawing, the “line connecting the hip joints” is oblique, which showsthat the user has properly rotated the hips at the time when the userhas put the right foot on the ground.

FIG. 16 is a diagram showing an example output in a case where, in thesecond embodiment, the terminal apparatus 10 outputs a measurementresult of the walking movement of the user and an evaluation result ofthe walking movement of the user based on the measurement result. Thiscorresponds to the processing in steps S1407 and S1409 shown in FIG. 14.

In the example in FIG. 16, the terminal apparatus 10 displays themeasurement result and the evaluation result for one cycle of thewalking movement of the user in such a manner that the results can becompared with a reference example (a model example or an ideal example)of the good walking posture. In the example output shown in FIG. 16,again, the terminal apparatus 10 may output the results by printing theresults on a paper medium with a printer or the like.

On the display 132, the terminal apparatus 10 divides one cycle of thewalking movement into a plurality of steps, and displays, for each step,an ideal model of the good walking posture, a measurement result of thewalking movement of the user, and an evaluation result of themeasurement result.

In the example in FIG. 16, one cycle of the walking movement is dividedinto a first step of “stepping a foot forward”, a second step of“starting to put his or her weight onto the front foot (starting to reston the front foot)”, and a third step of “completely putting his or herweight on the front foot (completely resting on the front foot)”.

Items evaluated by the terminal apparatus 10 in the first step, thesecond step, and the third step include items common to these steps anditems that are not common to these steps.

(1) Evaluation Items in the First Step

-   -   Inclination of the upper part of the body in the walking        movement: whether a part (such as the head or a shoulder) of the        upper part of the body is inclined forward or rearward with        respect to the position of the hips. For example, whether the        angle of inclination of a line defined by the position of the        pelvis, the position of the thoracic and lumbar spines, and the        position of the center between the shoulders with respect to the        vertical axis falls within a certain range or not.    -   Distance between the feet in the anteroposterior direction when        the user steps a foot forward: the degree of distance between        the feet in the anteroposterior direction. For example, whether        the distance between the ankle of the front foot and the ankle        of the back foot is equal to or greater than a certain distance        or not.    -   Position of the hips when the user steps a foot forward: whether        the hips are left behind or not when the user steps a foot        forward. For example, whether the position of the pelvis is        located forward or rearward of the shoulders or the thoracic and        lumbar spines.

(2) Evaluation Items in the Second Step

-   -   Inclination of the upper part of the body at the time when the        user has put the front foot on the ground: whether a part (such        as the head or a shoulder) of the upper part of the body is        inclined forward or rearward with respect to the position of the        hips.    -   Bending of the knee at the time when the user has put the front        foot on the ground: whether a part (such as the ankle, the knee,        or the pelvis) associated with the front foot is substantially        bent or not substantially bent. For example, the angle formed by        the line connecting the ankle and the knee and the line        connecting the knee and the pelvis falls within a certain range        of angles or not.

(3) Evaluation Items in the Third Step

-   -   Inclination of the upper part of the body at the time when the        user completely puts his or her weight on the front foot:        whether a part (such as the head or a shoulder) of the upper        part of the body is inclined forward or rearward with respect to        the position of the hips.    -   Extension of the pivot leg at the time when the user completely        puts his or her weight on the front foot: provided that the        front foot having landed on the ground after being stepped        forward is the pivot foot, whether a part (such as the ankle,        the knee, or the pelvis) of the pivot leg is extended or bent        when the user puts his or her weight on the pivot foot. For        example, the angle formed by the line connecting the ankle and        the knee and the line connecting the knee and the pelvis falls        within a certain range of angles or not.

Specifically, the terminal apparatus 10 displays an ideal model for thefirst step in a display part 132P11. In a display part 132P21, theterminal apparatus 10 displays a measurement result of the body of theuser in the first step. In a display part 132P31, the terminal apparatus10 displays an evaluation result of the movement of the body of the userin the first step.

In display parts 132P11, 132P12, and 132P13, the terminal apparatus 10displays positions of parts (joints) of the body at the time when theseevaluation items are appropriate for the good walking posture. Forexample, in the display part 132P11, positions of parts of the body inthe case where the upper part of the body is not inclined, the frontfoot and the back foot are at a certain distance (that is, the steplength is not too small), and the position of the hips is appropriate(that is, the hips are not left behind) are shown.

In the example shown, in the display part 132P31, as evaluation resultsof the evaluation item “inclination of the upper part of the body”, anevaluation result RF_ic of the right foot at the time when the usersteps the right foot forward and an evaluation result LF_ic of the leftfoot at the time when the user steps the left foot forward are displayedand superimposed on a predetermined bar. The predetermined bar is drawnin accordance with the possible range of the evaluation results. Theterminal apparatus 10 displays the predetermined bar divided into aregion indicating that the evaluation result is good and a regionindicating that the evaluation result is not good. In the example shown,the region indicating that the evaluation result is good is highlighted.For the evaluation item “inclination of the upper part of the body”, theterminal apparatus 10 displays the evaluation result RF_ic of the rightfoot and the evaluation result LF is of the left foot in associationwith each other in the “good” range, which indicates that the upper partof the body is not excessively inclined forward or rearward.

In the display part 132P31, as evaluation results, the terminalapparatus 10 also displays an evaluation result of the right foot and anevaluation result of the left foot for the evaluation items “distancebetween the feet in the anteroposterior direction” and “position of thehips” in association with a predetermined bar.

In the display part 132P12, the terminal apparatus 10 displays an idealexample for the second step. In a display part 132P22, the terminalapparatus 10 displays a measurement result of the body of the user inthe second step. In a display part 132P32, the terminal apparatus 10displays an evaluation result of the movement of the body of the user inthe second step.

In the display part 132P13, the terminal apparatus 10 displays an idealexample for the third step. In a display part 132P23, the terminalapparatus 10 displays a measurement result of the body of the user inthe third step. In a display part 132P33, the terminal apparatus 10displays an evaluation result of the movement of the body of the user inthe third step.

FIGS. 17A and 17B are diagrams showing examples where the terminalapparatus 10 outputs an evaluation result of the load on the knees inthe walking movement of the user. This corresponds to step S1409 in FIG.14.

FIG. 17A shows an example where measurement results of adduction andabduction of the hip joints in the walking movement of the user and anevaluation result thereof are output. The terminal apparatus 10 displaysthe measurement results of the walking movement of the user viewed alongthe anteroposterior axis (viewed from the front) on the display 132, sothat the user can easily check the bending of the knees in the walkingmovement.

In a display part 132P41, the terminal apparatus 10 displays measurementresults of the walking movement. In a display part 132P42, the terminalapparatus 10 displays a measurement result at the time when the userwalks by using the right foot as the pivot foot. In a display part132P43, the terminal apparatus 10 displays a measurement result at thetime when the user walks by using the left foot as the pivot foot.

In a display part 132P44, the terminal apparatus 10 displays apreferable type of measurement result (“pivot leg and swing leg arestraight”) and a type of measurement result with which the user may hurthis or her knee (“outside of knee may be hurt” and “inside of knee maybe hurt”), for information for the user. That is, both excessiveadduction and excessive abduction in the walking movement may hurt theknees in the walking movement.

In this way, the user can be prompted to avoid a walking movement thatmay hurt his or her knee, by referring to the output result of theterminal apparatus 10.

FIG. 17B is a diagram showing an example where an evaluation result ofwhether the knee is forward of the ankle or not in the walking movementof the user. The terminal apparatus 10 displays measurement results ofthe walking movement of the user viewed along the transverse axis(viewed from the side) on the display 132, so that the user can easilycheck whether the keens of the user are put too forward in the walkingmovement.

In a display part 132P51, the terminal apparatus 10 displays measurementresults of the walking movement. In a display part 132P52, the terminalapparatus 10 displays a measurement result at the time when the userwalks by using the right foot as the pivot foot. In a display part132P53, the terminal apparatus 10 displays a measurement result at thetime when the user walks by using the right foot as the pivot foot. Inthe display part 132P51, the terminal apparatus 10 displays that “loadon front knee is small” as an evaluation result of the walking postureof the user.

In a display part 132P54, the terminal apparatus 10 displays apreferable type of measurement result (“good”) and a type of measurementresult with which the user may hurt his or her knee (“knee is put tooforward”), for information for the user. That is, if the knee is forwardof the ankle when the user steps the front foot forward, the knee may behurt. The terminal apparatus 10 determines whether or not the knee isput too forward when the user steps the front foot forward based onwhether or not the angle formed by the line connecting the position ofthe ankle and the position of the knee and the line connecting theposition of the knee and the position of the pelvis is equal to orgreater than a predetermined angle (whether the part from the knee tothe ankle is put too forward or not).

As described above, the walking movement of the user is evaluated bydividing the walking cycle into a plurality of phases, such as a phaseof starting to step the pivot foot forward, a phase of putting the pivotfoot on the ground, and a phase of starting to put his or her weight onthe pivot foot. In this way, the user is prompted to improve the walkingmovement by referring to the evaluation result for each phase. Inaddition, since the user can refer to an ideal example of the movementof the body and compare the measurement result of the movement of thebody of the user with the ideal example, the user is prompted to bringhis or her walking posture closer to the good walking posture.

Third Embodiment

The measurement system 1 according to the third embodiment will bedescribed. In the third embodiment, a process will be described in whicha measurement result of the balance of the trunk of the user isdisplayed so that the user can achieve a posture in which the trunk isbalanced.

<Operation>

In the following, a process in which the terminal apparatus 10 measuresthe movement of the body of the user and makes an evaluation based onthe measurement result in the third embodiment will be described.

FIG. 18 is a flowchart showing a process which the terminal apparatus 10measures and evaluates the balance of the trunk of the user in the thirdembodiment.

In step S1803, the terminal apparatus 10 outputs a guide (video oraudio) for measuring the balance of the trunk (the shoulders, the hips,and the legs) of the user based on the measurement result from themeasurement apparatus 31. The terminal apparatus 10 displays measurementresults of the positions of the shoulders, the hips, and the legs of theuser viewed along the vertical axis with circles. The terminal apparatus10 outputs a guide to prompt the user to balance the trunk in accordancewith the degree of overlapping of the positions of the shoulders, thehips, and the legs.

In step S1805, the terminal apparatus 10 evaluates the balance of thetrunk of the user based on the measurement result from the measurementapparatus 31, and outputs an evaluation value. The terminal apparatus 10transmits the measurement result from the measurement apparatus 31 andthe evaluation result to the server 20.

In step S1853, the server 20 updates the measurement result database 287and the evaluation result database 288 based on the measurement resultand the evaluation result of the balance of the trunk of the user.

<Example Outputs>

FIGS. 19A, 19B, and 19C are diagrams showing examples in which theterminal apparatus 10 outputs a measurement result and an evaluationresult of the balance of the trunk of the user in the third embodiment.

FIG. 19A shows an example screen that outputs a measurement result inreal time in a situation where the terminal apparatus 10 measures thebalance of the trunk of the user with the measurement apparatus 31. Thiscorresponds to the processing in step S1803 in FIG. 18. In the examplein FIG. 19A, in a display part 132P61, a circular figure (a perfectcircle or an ellipse) corresponding to each of the shoulders, the hips,and the legs of the user is displayed.

In a display part 132P62, the terminal apparatus 10 displays ameasurement result of each part of the body of the user viewed along theanteroposterior axis.

In a display part 132P63, the terminal apparatus 10 displays thepositions of the parts, such as the shoulders, the hips, and the legs,of the body of the user viewed along the vertical axis based on themeasurement result from the measurement apparatus 31. The terminalapparatus 10 displays the measurement result from the measurementapparatus 31 in the display part 132P63 as required. Therefore, if thebody of the user fluctuates, the circular figures corresponding to theparts, such as the shoulders, the hips, and the legs, are not alignedand do not overlap with each other. In the example screen in FIG. 19A,the parts of the body of the user are displaced and not aligned alongthe vertical axis. The user is prompted to move his or her body tobalance the trunk by watching the display in the display part 132P63 ofthe display 132.

In a display part 132P64, the terminal apparatus 10 makes a notificationto prompt the user to achieve a posture in which the trunk is balanced.Based on the measurement result from the measurement apparatus 31, theterminal apparatus 10 determines a movement of the body of the userrequired to balance the parts of the body of the user. In the exampleshown, the positions of the head and the legs are located rearward ofthe position of the hip part of the user. That is, the hips of the userstick out forward. In the display part 132P64, the terminal apparatus 10prompts the user to correct the inclination of the body in theanteroposterior direction. In this way, the user can easily recognizethe movement of the body required to achieve a posture in which thetrunk is balanced.

FIG. 19B shows an example display of a measurement result in a statewhere the trunk of the user is balanced. In the display part 132P63, theparts, such as the shoulders, the hips, and the legs, of the user arealigned along the vertical axis, and the trunk is balanced. In thedisplay part 132P64, the terminal apparatus 10 notifies the user thatthe trunk is balanced.

FIG. 19C shows an example display in a case where the measurement of thebalance of the trunk is finished, and an evaluation result is output.This corresponds to the processing in step S1805 in FIG. 18.

In a display part 132P65, the terminal apparatus 10 displays that themeasurement of the balance of the trunk is finished, and outputs anevaluation result based on the measurement result.

Based on the evaluation result of the balance of the trunk, the terminalapparatus 10 determines a training regimen for balancing the trunk, as atraining regimen to be proposed to the user. In the example shown, inthe display part 132P65, the terminal apparatus 10 displays theevaluation result of the balance of the trunk of the user, and acandidate of the training regimen proposed to the user based on theevaluation result. In a display part 132P66, the terminal apparatus 10receives an operation for starting the training regimen proposed to theuser. In response to an input operation by the user in the display part132P66, the terminal apparatus 10 reads the corresponding trainingregimen.

In a display part 132P67, the terminal apparatus 10 receives a choice ofa training regimen from the user. That is, in the display part 132P67,the terminal apparatus 10 receives an input operation for starting othertraining than the training regimen proposed to the user by the terminalapparatus 10 based on the evaluation result of the balance of the trunk.

The details of the embodiments described above may be combined with eachother.

Fourth Embodiment

In a fourth embodiment, a technique of proposing a training regimenappropriate to the user based on the result of measurement of themovement of the body of the user, such as the ROM check (firstembodiment), the check of the walking posture (second embodiment), andthe measurement of the balance of the trunk (third embodiment), will bedescribed. By measuring and evaluating the movement of the body of theuser as described above, the terminal apparatus 10 can determine a partof the body of the user to be trained, and proposes an appropriatetraining regimen in accordance with the determination result.

With such a configuration, when adding a content of a training regimen,the content of the training regimen can be associated with a part of thebody to which the content is appropriate and a possible trouble of thebody of the user that can be solved by the content, so that contentssuitable for the measurement result of the movement of the body of theuser can be provided to the user. Therefore, contents can be more easilyadded to the training regimen.

(1) As described above, the terminal apparatus 10 may propose anexercise regimen to the user based on the measurement result and theevaluation result of each step of the walking cycle of the user. Forexample, the terminal apparatus 10 compares the movement of the body ofthe user with an ideal example of the movement of the body (as in theexample in FIG. 16), and determines a muscle that is desirably improvedby the user. For example, if the distance between the feet in theanteroposterior direction is small in the evaluation result of the firststep (a movement of stepping a foot forward) in the walking movement ofthe user, the server 20 determines an exercise regimen that can beexpected to increase the step length by referring to the training iteminformation 285. That is, the server 20 may retain exercise regimens tobe proposed in association with the evaluation results of evaluationitems for each step of the walking movement and evaluation items for thebalance of the trunk in advance.

(2) The terminal apparatus 10 may propose a training regimen that caneliminate a left-right difference of the movement of the body of theuser to the user. As described above, the terminal apparatus 10evaluates the left-right balance of the movement of the body of the userby the ROM check and the walking posture check. For example, theterminal apparatus 10 evaluates the left-right balance of the rotationalmovement about the vertical axis based on the evaluation results of therotation of the shoulders and the rotation of the hips in the walkingmovement of the user. The terminal apparatus 10 determines a muscle thatcauses the left-right asymmetry of the movement of the body of the user.For example, the terminal apparatus 10 retains information on a musclethat causes a left-right difference of the amount of rotation of theshoulders in advance. By retaining items of evaluation in the walkingmovement and information on muscles that cause left-right differences inassociation with each other in advance, the terminal apparatus 10 candetermine the muscle to be trained in order to eliminate a left-rightimbalance.

(3) Based on the evaluation result of the movement of the body of theuser, the terminal apparatus 10 may predict a possible future state ofthe body of the user, and propose a training regimen to the user basedon the prediction result.

For example, as predetermined information, the terminal apparatus 10stores (i) items “stiff neck or shoulder”, “flabby upper arm”, “hunchedshoulders or posture with the head inclined forward”, “lower back pain”,“swelling of the legs”, “arm muscle stiffness” and the like and (ii)information on parts of the body of the user in association with eachother. The terminal apparatus 10 also stores information on trainingregimens to be proposed for training in association with the possibletrouble items of the body of the user described above.

The terminal apparatus 10 performs the ROM check and the walkingmovement check and determines a muscle that is desirably improved basedon the measurement result and the evaluation result thereof. Forexample, in a case that the ROM check shows that twisting of the body isnot good, the terminal apparatus 10 retains information on what musclesare associated with the movement in advance. For example, for the ROMcheck, the terminal apparatus 10 retains information on muscles inassociation with evaluation items of the range of motion of theshoulders, the hips and the like.

The terminal apparatus 10 determines a possible future trouble byreferring to the predetermined information described above by locating astiff muscle, for example. Based on the determination result, theterminal apparatus 10 determines a training regimen to be proposed tothe user so as to solve the possible future trouble of the user.

As described above, evaluation items of the balance of the walkingmovement and the twisting of the body are associated with information onmuscles, and the information on muscles and information on possibletrouble items of the body of the user are associated with each other.Therefore, the terminal apparatus 10 can propose a training regimencustomized for the user to the user based on the evaluation result ofthe movement of the body of the user.

<Data Structure in Fourth Embodiment>

FIG. 20 is a diagram showing a data structure of various data stored inthe server 20 in the fourth embodiment.

The server 20 retains an item “possible future trouble” and an item“associated muscle” for each measurement item in the measurement iteminformation 282.

The item “possible future trouble” is information (such as a tag “stiffshoulder” or “stiff neck”) for identifying a trouble that may occur inthe body of the user in future depending on the evaluation result of themeasurement result for a measurement item.

The item “associated muscle” is information on a part (muscle) of thebody that is desirably trained depending on the evaluation result of themeasurement result for a measurement item. In the item “associatedmuscle”, a plurality of pieces of information on a part of the bodyranked according to the evaluation result may be retained.

The server 20 retains an item “content”, an item “trouble to be solved”,and an item “associated muscle” for each training item in the trainingitem information 285.

The item “content” indicates a content (such as a movie, referencemotion data for the movement of the body of the user, or audio data) tobe presented to the user when performing a training.

The item “trouble to be solved” is information (such as a tag “stiffshoulder” or “stiff neck”) for identifying a trouble that may occur inthe body of the user in future depending on the evaluation result.

The item “associated muscle” is information on a part (muscle) of thebody that is desirably trained depending on the evaluation result for atraining item. In the item “associated muscle”, a plurality of pieces ofinformation on a part of the body ranked according to the evaluationresult may be retained.

The server 20 retains an item “possible future trouble” in theevaluation result database 288. The item “possible future trouble” isinformation (such as a tag “stiff shoulder” or “stiff neck”) foridentifying, for an evaluation result, a trouble that may occur in thebody of the user in future depending on the evaluation result.

As described above, the server 20 updates the evaluation result database288 in accordance with the evaluation result of the movement of the bodyof the user. For example, information on a trouble that may occur in thebody of the user in future can be located based on the measurementresult (measurement item information 282) of the ROM check. The server20 locates information on a possible future trouble by referring to theevaluation result database 288, and extracts a training item to beproposed to the user by referring to the training item information 285.When extracting a plurality of training items to be proposed to theuser, the server 20 assigns priorities according to a predeterminedrule. For example, based on the badness of the evaluation result of themovement of the body of the user, the server 20 may assigns a higherpriority to a training item that can solve a trouble corresponding to abad evaluation result.

<Example Screens in Fourth Embodiment>

FIGS. 21A, and 21B show example screens in a situation where theterminal apparatus 10 presents a possible future trouble of the body ofthe user and proposes an exercise regimen corresponding to the troublebased on the measurement result and evaluation result of the movement ofthe body of the user.

In FIG. 21A, the terminal apparatus 10 displays a result ofidentification of possible troubles of the body of the user based on themeasurement result and evaluation result of the movement of the body ofthe user.

The terminal apparatus 10 prompts the user to perform measurements everyday with the terminal apparatus 10 in a display part 132P71, anddisplays a condition that may occur in the body of the user in future ina display part 132P72. In the example shown, the terminal apparatus 10displays that a trouble “stiff neck or shoulder” may occur in anoperation receiving part 132P73, and displays a training regimencorresponding to the trouble “stiff neck or shoulder” in response to auser input operation in the operation receiving part 132P73. Theterminal apparatus 10 also displays that a trouble “flabby upper arm”may occur in an operation receiving part 132P74, and displays a trainingregimen corresponding to the trouble “flabby upper arm” in response to auser input operation in the operation receiving part 132P74. Theterminal apparatus 10 also displays that a trouble “hunched shoulders orposture with the head inclined forward” may occur in an operationreceiving part 132P75, and displays a training regimen corresponding tothe trouble “hunched shoulders or posture with the head inclinedforward” in response to a user input operation in the operationreceiving part 132P75.

In a display part 132P76, the terminal apparatus 10 displays anoperation receiving part 132P77 for starting less training regimens thanthose displayed in the display part 132P76 for a user who doesn't havemuch time, from among the training regimen candidates to be proposed tothe user. The terminal apparatus 10 assigns priorities to the trainingregimen candidates to be proposed to the user according to apredetermined rule and displays the training regimen candidates in thedisplay part 132P76.

In a display part 132P78, the terminal apparatus 10 receives anoperation for editing a training regimen to be performed, such as“favorite”, from the user.

In response to receiving an input operation in an operation receivingpart 132P91, the terminal apparatus 10 presents a questionnaire to theuser, and receives a response to the questionnaire. Based on the inputresult to the questionnaire, the terminal apparatus 10 may determine atrouble of the user, and propose a training regimen corresponding to thedetermine trouble.

FIG. 21B shows a situation where a recommended training regimen isproposed to the user. In a display part 132P81, the terminal apparatus10 displays, to the user, information on a part of the body (such as “Doyou have a stiff neck or shoulder?”) concerning the trouble determinedbased on the measurement result and evaluation result. In the displaypart 132P81, the terminal apparatus 10 also displays a cause of thetrouble of the body of the user (such as “Your shoulders tend to behunched.”) based on the measurement result and evaluation result. In thedisplay part 132P81, the terminal apparatus 10 also displays a guidelinefor solving the trouble of the body of the user (such as “guidelineexpected to improve the blood flow in the upper part of the body” or“guideline for stretching the hamstrings”) based on the measurementresult and evaluation result, and proposes an exercise corresponding tothe guideline.

For example, in response to an input operation in the operationreceiving part 132P73, the terminal apparatus 10 displays operationreceiving units 132P83 to 132P87 for starting a recommended training ina display part 132P82. In response to an input operation in any of theseoperation receiving units 132P83 to 132P87, the terminal apparatus 10may reproduce a movie for starting a training, for example.

In response to an input operation in the operation receiving part132P77, the terminal apparatus 10 narrows down the number of trainingregimens to be proposed for a user who doesn't have much time anddisplays operation receiving units 132P83 to 132P85 in a display part132P88.

In this way, the user can be prompted to continue training according tothe condition of the user.

Example Operation

Operations of the apparatuses in the fourth embodiment described abovewill be described.

FIG. 22 shows a flow of processing performed by the apparatuses in thefourth embodiment.

In step S2203, the terminal apparatus 10 evaluates at least any of thejoint range of motion, the walking posture, and the balance of the trunkof the user based on the measurement result of the movement of the bodyof the user, and outputs the evaluation result. The terminal apparatus10 transmits the measurement result and the evaluation result to theserver 20.

In step S2253, the server 20 updates a database (such as the evaluationresult database 288) based on various measurement results of the user,such as the measurement result and the evaluation result received fromthe terminal apparatus 10.

In step S2255, the server 20 determines a possible future trouble of thebody of the user based on the evaluation result database 288 or thelike. The server 20 selects a training regimen relating to the part ofthe body associated with the determined trouble from among a pluralityof training regimens based on the training item information 285, andtransmits the selected training regimen back to the terminal apparatus10 of the user.

In step S2205, the terminal apparatus 10 displays the training regimencorresponding to the possible future trouble of the body of the user.

Example Modification

In the fourth embodiment, a question may be presented to the user, andpriorities may be assigned to the training regimens proposed to the useraccording to the response (questionnaire result) input by the user.

In this way, not only does the server 20 propose a training regimenbased on the measurement result of the body of the user, but also theserver 20 can present a question concerning the body to the user andpropose a training regimen based on the response to the question fromthe user. That is, the server 20 updates the proprieties of the trainingregimens proposed to the user based on the response to the question. Theserver 20 presents a training regimen having the highest priority forthe user to the user based on not only the evaluation result of the bodybut also the response to the question from the user. In this way, theserver 20 can propose a training regimen that is more appropriate to thecondition of the body of the user.

For example, the terminal apparatus 10 can receive information on amuscle of the body of the user that has not been measured yet by themeasurement unit 30 of the measurement system 1 (that is, that has notbeen evaluated based on the measurement result yet) by receiving aquestionnaire response concerning the body of the user. The terminalapparatus 10 can also determine a possible future trouble relating to amuscle that has not been evaluated yet by the measurement system 1,based on a questionnaire result input by the user. The terminalapparatus 10 determines a training regimen to be proposed to the user soas to properly cope with a possible future trouble of the body of theuser, based on the trouble determined based on the measurement result ofthe movement of the body and the response to the questionnaire from theuser.

<Data Structure in Example Modification>

FIG. 23 is a diagram showing data structures of various data stored inthe server 20 in the modification of the fourth embodiment.

The server 20 retains an item “response option”, an item “currenttrouble”, an item “possible future trouble”, and an item “associatedmuscle” for each questionnaire result in user questionnaire information289.

The item “response option” includes information on options for the userof the terminal apparatus 10 to respond to a question (questionnaire)presented to the user. For example, when the question concerns physicalcomplaints of the user, parts that can be involved with physicalcomplaints are presented as options to the user. Then, based on theresult input for each option by the user, a trouble of the user relatingto a part of the body can be determined.

The item “current trouble” is information (such as a tag “lower backpain” or “knee pain”) for identifying, for a questionnaire response, atrouble the user may currently have depending on the evaluation resultof the questionnaire response.

The item “possible future trouble” is information (such as a tag “stiffshoulder” or “stiff neck”) for identifying, for a questionnaireresponse, a trouble that may occur in the body of the user in futuredepending on the evaluation result of the questionnaire response.

The item “associated muscle” is information, for a questionnaire result,on a part (muscle) of the body that is desirably trained depending onthe evaluation result of the questionnaire response. In the item“associated muscle”, a plurality of pieces of information on a part ofthe body ranked according to the response may be retained.

The server 20 retains an item “current trouble” in the evaluation resultdatabase 288. The item “current trouble” is information (such as a tag“lower back pain” or “knee pain”) for identifying, for an evaluationresult, a trouble the user may currently have depending on theevaluation result of the questionnaire response.

<Operation in Example Modification>

As described above, the server 20 updates the evaluation result database288 in accordance with the evaluation result of the movement of the bodyof the user and the user questionnaire information 289. For example,information on a trouble that may occur in the body of the user infuture can be located based on the measurement result (measurement iteminformation 282) of the ROM check. For example, a trouble the usercurrently has or may have in the near future can be determined based onthe questionnaire result concerning the body from the user (userquestionnaire information 289). The server 20 locates information on apossible future trouble of the user by referring to the evaluationresult database 288, and extracts a training item to be proposed to theuser by referring to the training item information 285.

When extracting a plurality of training items to be proposed to theuser, the server 20 assigns priorities according to a predetermined rulebased on the badness of the evaluation result of the movement of thebody of the user. For example, based on the badness of the evaluationresult of the movement of the body of the user, the server 20 may assigna higher priority to a training item that can solve a troublecorresponding to a bad evaluation result. The server 20 modifies andsets priorities of the training items to be proposed to the user basedon the questionnaire result. For example, based on the badness of theevaluation result of the body relating to the questionnaire responsecompared with the evaluation result of the ROM check, the server 20 mayassign a higher priority to a training item that can solve the troubleof the body of the user determined based on the questionnaire result.

FIG. 24 is a diagram showing a flow of an operation in the examplemodification of the fourth embodiment.

As shown in FIG. 24, at any timing, such as in step S2202 or step S2204,the terminal apparatus 10 presents a question (questionnaire) to theuser of the terminal apparatus 10.

The processing in step S2202 will be described. The terminal apparatus10 displays a question on the display 132 in response to the usercompleting the processing of logging in the measurement system 1 in stepS700, for example. The terminal apparatus 10 receives an input of aresponse to the question from the user. The terminal apparatus 10transmits the input response to the server 20.

In step S2252, the server 20 updates the user questionnaire information289 based on the information received from the terminal apparatus 10.

In step S2253, the server 20 updates various databases, such as themeasurement result database 287, in response to receiving themeasurement result from the measurement unit 30 or the like and theevaluation result based on the measurement result that are transmittedfrom the terminal apparatus 10 in step S2203.

The processing in step S2204 will be described. The terminal apparatus10 presents a question to the user (by displaying on the display 132,for example) in response to the measurement of the movement of the bodyof the user being completed in step S2203. The terminal apparatus 10transmits an input response to the server 20.

In step S2254, the server 20 updates the user questionnaire information289 based on the information received from the terminal apparatus 10.

In step S2256, the server 20 determines a current trouble of the user ora future possible trouble of the body of the user based on the userquestionnaire information 289, the evaluation result database 288 or thelike. The server 20 selects a training regimen corresponding to thedetermined trouble from among a plurality of training regimens based onthe training item information 285, and transmits the selected trainingregimen back to the terminal apparatus of the user. For example, theserver 20 determines a trouble of the user based on the user response toa question (user questionnaire information 289), and determines atraining regimen to be proposed to the user according to the determinedtrouble (training item information 285). The server 20 also determines atraining regimen to be proposed to the user based on the evaluationresult database 288, as described above. The server 20 transmits one ormore training regimens determined in this way back to the terminalapparatus 10. In this processing, the server 20 may assign priorities toa plurality of determined training regimens according to a predeterminedrule, and transmit a training regimen back to the terminal apparatus 10according to the priority.

Example Screen 1 in Fourth Embodiment

FIGS. 25A and 25B show example screens for the user to respond aquestionnaire in a situation where the terminal apparatus 10 presents apossible trouble of the body of the user based on the measurement resultand evaluation result of the movement of the body of the user, andproposes an exercise regimen corresponding to the trouble. For example,the terminal apparatus 10 displays the screen (display part 132P71)shown in FIG. 21A by accessing the server 20, for example.

At a timing when the screen is displayed (a timing when the terminalapparatus 10 receives information for displaying the screen from theserver 20 and displays the screen on a browser or the like), at a timingwhen a predetermined time has lapsed since the screen was displayed, orin response to a predetermined condition, such as that the user hasperformed a predetermined operation on the screen, being satisfied, theterminal apparatus 10 displays a display part 132PA2 shown in FIG. 25Ato prompt the user to respond the questionnaire.

FIG. 25A shows the display part 132PA2 for the user to respond thequestionnaire by overlaying the display part 132PA2 on the screen(display part 132P71 described above with reference to FIG. 21A) inwhich the result of determination of a possible trouble of the body ofthe user based on the measurement result and evaluation result of themovement of the body of the user.

In the display part 132PA2, the terminal apparatus 10 displays aquestion text (“What kind of trouble do you have?”) in a display part132PA3.

The server 20 determines the details of the question text presented tothe user in the display part 132PA3. For example, the server 20 retainsa plurality of question texts in advance, and retains information onparts of the body of the user and information on candidates of theresponse to the question text in association with each question text.The server 20 may refer to the measurement result database 287 and theevaluation result database 288 to determine a question text to bepresented to the user based on the history of the measurements performedby the user, and make the terminal apparatus 10 display the determinedquestion text. For example, based on the history of the measurementsperformed by the user with the measurement system 1 (such as the timeintervals between the measurements), the server 20 may present the usera question text inquiring whether the user is aware of a disorder of apart of the body (such as “What kind of trouble do you have?”) when acertain period has elapsed since the user performed the lastmeasurement.

Furthermore, the server 20 may determine a part of the body included inthe question text to be presented to the user based on the measurementresults of parts of the body of the user, for example. For example, whena part of the body of the user has not been measured at least for acertain period (when there is no history of measurements of the hips orwhen a certain period has elapsed since the last measurement, forexample), the part of the body may be included in the question text(such as “How is your lower back?”).

In the example in FIG. 25A, a display part 132PA4 displays candidates ofthe response to the question text and receives a selection from theuser. In the example shown, in the display part 132PA4, as candidates ofthe response to the question text, “lower back trouble” for selecting adisorder of the lower back, “knee trouble” for selecting a disorder ofthe knee, and “***” for requesting display of response candidatesrelating to the other parts of the body are displayed. The terminalapparatus 10 receives a selection of “lower back trouble” from the userand highlights the candidate so that the user can easily recognize theselection by the user.

In a display part 132PA5, the terminal apparatus 10 displays a questiontext concerning the degree of the trouble (degree of the disorder of thepart of the body).

In a display part 132PA6, the terminal apparatus 10 displays candidatesof the response to the question text displayed in the display part132PA5, and receives a selection from the user. In the example shown, inthe display part 132PA6, the terminal apparatus 10 displays “mild”,which means that the degree of the trouble is mild, “moderate”, whichmeans that the degree of the trouble is moderate, and “severe”, whichmeans that the degree of the trouble is severe, as options. The terminalapparatus 10 receives a selection of the option “moderate” from theuser, and highlights the selected option.

In a display part 132PA1, the terminal apparatus 10 receives anoperation that indicates that the response to the question text iscompleted from the user. In response to this, the terminal apparatus 10transmits the response of the user to the server 20. The server 20receives the response of the user, and updates the user questionnaireinformation 289. For example, when the user selects information on apart of the body, such as “lower back trouble”, as the response, theserver 20 updates the user questionnaire information 289 based on thetrouble of the user “lower back trouble”.

FIG. 25B shows an example screen in a situation where a training regimenis proposed to the user based on the response to the question text fromthe user and the measurement result and evaluation result of a part ofthe body of the user.

In the display part 132P81, the server 20 displays information on atrouble of a part of the body of the user (such as information includinga part of the body and the degree of the trouble, such as “the load onyour knees seems to be moderate”) based on the response to the questiontext from the user (user questionnaire information 289). In the displaypart 132P81, the terminal apparatus 10 also displays a cause of thetrouble of the body of the user (such as “Do you keep an uncomfortableposture for a long time?”). In the display part 132P81, the terminalapparatus 10 also displays a guideline for solving the trouble of thebody of the user (such as “loose the muscles around the lower back”),and proposes an exercise corresponding to the guideline.

Other Example Modifications

(1) The user is questioned about exercise experience.

In the above description of the embodiment, examples have been describedwhere the server 20 presents a question text to the user and proposes atraining regimen based on the response to the question. Alternatively,the server 20 may question the user about exercise experience in thequestion presented to the user.

For example, the server 20 may presents a question text concerning hisor her past sports experience or his or her exercise habit. The server20 receives a response of the user to the question (such as a sportplayed by the user in the past or the presence or absence of exercisehabit). Based on the response of the user, when the user has relativelylittle exercise experience (the period for which the user played sportsis equal to or less than a certain period, or the frequency of exerciseis equal to or less than a predetermine frequency, for example), theserver 20 may propose, to the user, a training item the load of which onthe body is relatively small or the difficulty of which is relativelylow, from among the training items included in the training iteminformation 285. For example, for each training item in the trainingitem information 285, information on the difficulty (if a training canbe performed by a person unfamiliar with exercise without any problem,the difficult of the training is low, for example) and the load on thebody (different methods of training for different parts (such as upperarm) of the body have different loads on the body, for example) isretained in advance. In this way, a training regimen can be presentedbased on the exercise experience of the user.

In the example described above, a training regimen that is relativelyeasy to accomplish is proposed to a user unfamiliar with exercise, sothat the user is more likely to feel a sense of accomplishment whenperforming the training regimen. In this way, the user can be motivatedto continue training.

(2) Whether the user is familiar with exercise is evaluated based on themeasurement result of the movement of the body of the user.

Furthermore, the server 20 may evaluate the exercise experience of theuser based on the measurement result of the movement of the body of theuser by the measurement unit 30, and determine a training regimen to bepresented to the user based on the evaluation result.

For example, the server 20 retains, in advance, a pattern of themeasurement result of one who is unfamiliar with exercise and a patternof the measurement result of one who is familiar with exercise inassociation with the measurement result of the movement of the body ofthe user, such as the joint range of motion, the walking posture, or thebalance of the trunk. Based on the measurement result of the movement ofthe body of the user, the server 20 evaluates the degree of familiarityof the user with exercise. For example, the server 20 compares themeasurement result of the movement of the body of the user with thepatterns described above to evaluate the degree of agreement, andoutputs an evaluation value indicating whether the user is familiar withexercise or not. For example, the server 20 presents a body movementmodel to the user, and evaluates that the user is familiar with exerciseif the user can move his or her body by generally following the bodymovement model. If the measurement result of the movement of the body ofthe user deviates from the body movement model, audio may be output tothe user to eliminate the deviation. For example, when the position of ahand is different from that in the movement model, audio that promptsthe user to make the position of the hand agree with that in themovement model (for example, to move the hand in a predetermineddirection) may be output. When the movement of the body of the useragrees with the movement model (that is, the user is moving by properlyfollowing the movement model), audio or the like is output to indicatethat the movement of the body of the user is proper.

When measuring the movement of the body of the user, the server 20 mayoutput an evaluation value indicating whether the user is familiar withexercise or not based on whether the user is moving by properlyfollowing a video or audio guide presented to the user, for example. Forexample, when the user is moving by properly following an audio output,the server 20 may evaluate that the user is familiar with exercise.

Based on the evaluation value, the server 20 determines the trainingregimen to be proposed to the user.

As described above, the server 20 may retain a history of measurementresults from the measurement unit 30 in the measurement history database283, and display, to the user, the time and result of the evaluation ofwhether the user is familiar with exercise or not and the history of thetraining regimens proposed to the user. In this way, the user canconfirm that the user is becoming familiar with exercise and thetraining regimen is changing, and can have self-efficacy.

(3) The user is questioned about continuing training

Furthermore, the server 20 may present a question about continuingtraining to the user. For example, a question concerning the personalityof the user may be presented, such as whether the user has succeeded incontinuing training in the past or periods for which the user was ableto continue training.

Based on the response to the question, the server 20 determines atraining regimen to be presented to the user. For example, suppose thata problem that can occur in the body of the user is located, and atraining regimen relating to the part of the body associated with theproblem is presented to the user as described in the above fourthembodiment and the like. When the user responds with a negative responseconcerning whether the user can continue training (for example, the usertends to quit training), the server 20 may change the training regimento be proposed to the user according to the problem. For example, ifthere is a plurality of training regimens relating to the part of thebody associated with the problem, another training regimen is proposedto the user. As a result, it can be expected that the user can moreeasily train without getting bored and continue training.

Based on the measurement history database 283 or the like, the server 20determines the intervals of times when the user trained. That is, theserver 20 determines the frequency of training of the user. Based on thelength of the intervals, the server 20 may presents a question aboutcontinuing training to the user. For example, when the frequency oftraining is decreasing, the server 20 can propose another trainingregimen that can be performed by the user without getting bored, byreceiving a response to a question about continuing training from theuser. Furthermore, the server 20 may change the training regimen to beproposed to the user based on the length of the intervals. This canallow the user to train without getting bored and continue training.

Fifth Embodiment

<Overview>

In a fifth embodiment, a technique of allowing the user moving his orher body to more easily check whether the user is moving in an ideal wayor not will be described.

(1) Visual information on the movement of the body of the user whenmoving his or her body is reduced to facilitate the user to check his orher movement.

Specifically, when the user makes a particular movement (such as amovement for evaluating the movement of trunk muscles or the joint rangeof motion, such as lateral arm elevation, or a movement for strengthtraining), the movement of the body of the user is sensed as point clouddata, and an avatar corresponding to the user and an auxiliary linecorresponding to the movement of the body of the user are drawn based onthe point cloud data and presented to the user. This allows the user tomore easily check whether the user is moving parts of his or her body asintended.

(2) A video conferencing tool or the like is used to make it easier fora trainer to instruct a trainee online.

In the fifth embodiment, a situation will be described in which with thetechnique described above, a trainee (first user) who is instructed howto move his or her body and a trainer (second user) who instructs thetrainee how to move his or her body communicate with each other online.

Specifically, an avatar representing the movement of the body of thetrainee in the form of point cloud data is displayed on a terminal ofthe trainer, so that the trainer can check not only the taken image ofthe body of the trainee but also the movement of the avatar, whichprovides less visual information than the image. This allows the trainerto more easily advise the trainee how to move his or her body online.

That is, in the fifth embodiment, there will be described a series oftechniques for performing various kinds of processing, such as (1) afirst processor of a terminal of the trainee (first user) senses themovement of the body of the trainee (first user) as point cloud data bymeans of the measurement unit 30 or the like and (2) the first processortransmits information based on the point cloud data on the trainee(first user) (such as an image of the avatar of the trainee drawn basedon the point cloud data or the point cloud data itself) to a terminal ofthe trainer (second user).

Here, the terminal apparatus of the trainee (first user) that performsvarious kinds of processing, such as sensing of the movement of the bodyof the trainee (first user) as point cloud data, and transmission ofinformation based on the point cloud data on the trainee (first user) tothe terminal of the trainer (second user) by the first processorthereof, may be a terminal apparatus of the user, such as a computer, asmartphone, or a tablet terminal.

<Data Structure in Fifth Embodiment>

FIG. 26 is a diagram showing a data structure of a trainee informationdatabase 2601, a trainer information database 2602, and a traininghistory database 2603 stored in the server 20 in the fifth embodiment.

Although not shown, in addition to these databases, the server 20manages a database of information on the types of movements of the bodyof the user (such as a movement for evaluating the joint range of motionor the like, such as lateral arm elevation, a movement for strengthtraining, such as push-up, a prescribed body movement, such as yoga, orother movements), information on the types of auxiliary lines used whendisplaying the movement of the body of the user as an avatar in the formof a set of graphic elements based on point cloud data, and informationon positions of the auxiliary lines drawn (positions associated withparts of the body of the user represented by the avatar) associated witheach other.

The trainee information database 2601 is a database for managinginformation on the trainee user. The trainer information database 2602is a database for managing information on the trainer user. The traininghistory database 2603 is a database for managing a history of trainingsperformed by the trainee under the instruction of the trainer.

As shown in FIG. 26, the trainee information database 2601 includes anitem “trainee ID”, an item “name”, an item “age”, an item “sex”, an item“weight”, an item “body fat percentage”, an item “purpose”, an item“ideal figure”, an item “display on screen”, an item “physical fitnesslevel”, and an item “remarks”, for example.

The item “trainee ID” is information for identifying the user whoreceives training.

The item “name” is information indicating the name of the user whoreceives training.

The item “age” is information indicating the age of the user whoreceives training.

The item “sex” is information indicating the sex of the user whoreceives training.

The item “weight” is information indicating the weight of the user whoreceives training.

The item “body fat percentage” is information indicating the body fatpercentage of the user who receives training.

The item “purpose” is information indicating the purpose of the user whoreceives training. Specifically, the purpose of training of a useridentified by a user ID “U001” and a name “Mori” is “to get in shape”,and the purpose of training of a user identified by a user ID “U002” anda name “Sato” is “to lose weight”. In a situation, the user (trainer)who instructs the user (trainee) who receives training can present atraining regimen to the user (trainee) according to the purpose. Inanother situation, the terminal apparatus 10 or the server 20 maydetermine a training regimen to be presented to the user based on alearned model or the like described later.

The item “ideal figure” is information indicating an ideal figure of theuser who receives training. Specifically, the ideal figure of the useridentified by the user ID “U001” and the name “Mori” is “not specified”,and the ideal figure of the user identified by the user ID “U002” andthe name “Sato” is “00”. As the ideal figure, the user can specify anentertainer in real life or a fictitious existence (which may be acartoon character if the ideal figure of the user is the body shape ofthe character, for example). In another situation, the server 20 maydetermine and present a training regimen based on the “ideal figure” andthe “purpose”.

The item “display on screen” is information indicating the way ofdisplay of the user whose image is taken with a camera or the like. Theway of display may be as follows: an image of the user is taken with anRGB camera or the like, and the taken image generated by the camera isdisplayed as it is; or the movement of the body of the user is measuredas point cloud data by a sensor capable of distance measurement (such asa depth camera or light detection and ranging (LiDAR)), and an avatar isdisplayed based on the point cloud data. Specifically, the “display onscreen” of the user identified by the user ID “U001” and the name “Mori”is “avatar”, which means that when displaying a movie of the user takenwith a camera or the like on the screen, the taken movie of the user isnot displayed as it is, but an avatar (such as an avatar based on somecharacter) of the user created based on the taken movie is displayed.

In this way, demands of the user who receives training, such as that theuser does not want other persons to see his or her face or how he or shetrains in the first lesson or that the user wants to keep his or herprivacy, can be met, and the user can receive training with peace ofmind.

The item “physical fitness level” is information indicating the physicalfitness level of the user who receives training, which is a stepwiseevaluation of the exercise capacity of the user. Specifically, whensensing the movement of the body of the user who receives training, thephysical fitness level of the user, which can be a criterion fordetermining what kind of exercise the user can do, for example, isdetermined at the same time based on the information on the movement ofthe body of the user. The physical fitness level of the user identifiedby the user ID “U001” and the name “Mori” is “5”, and the physicalfitness level of the user identified by the user ID “U002” and the name“Sato” is “2”. As methods of determining the physical fitness level,after the movement of the body of the user is sensed, the terminalapparatus 10 or the server 20 may determine the physical fitness levelof the user based on a leaned model or the like, or the user (trainer)who instructs the user (trainee) who receives training may determine thephysical fitness level of the trainee based on the sensing.

In this way, the user can train according to a training regimenappropriate to the physical fitness level of the user, and therefore cancontinue training without any stress. In a case of a group training, theserver 20 can find users of similar physical fitness levels and make amatch between users (a trainer and trainees) so that the found users cantrain together. For example, when users (trainees) of different physicalfitness levels train together, the trainer gives different instructionsor different training regimens to different trainees, and some users maylose motivation. However, the physical fitness level can be prevented asdescribed above.

The item “remarks” is information indicating circumstances of the userwho receives training that are to be specially considered. Circumstancesto be specially considered include considerations concerning the safetyof training (such as the safety from the medical point of view or thesafety of the training regimen itself). For example, when the user istold by a doctor not to train hard because of his or her hypertension,it is entered in the remarks column. Based on the entries in the item“remarks”, the server 20 may determine an optimal training regimen forthe user who receives training. Furthermore, the terminal apparatus 10Bof the trainer may obtain information of the item “remarks” from theserver 20 and present the information to the trainee, thereby providinginformation useful for the trainer to determine the training regimen forthe trainee.

In this way, even if the user has a chronic disease, the server 20 orthe like can determine the training regimen to be presented to the userbased on the item “remarks”, or the trainer can select a trainingregimen based on the item “remarks”, so that the user can continuereasonable training with piece of mind.

The trainer information database 2602 includes an item “trainer ID”, anitem “name”, an item “sex”, an item “years of experience”, an item“specialty”, an item “required physical fitness level”, an item “lessontype”, and an item “remarks”, for example.

The item “trainer ID” is information for identifying the user whoinstructs the user who receives training.

The item “name” is information indicating the name of the user whoinstructs the user who receives training.

The item “sex” is information indicating the sex of the user whoinstructs the user who receives training.

The item “years of experience” is information indicating the years ofexperience of training instruction of the user who instructs the userwho receives training. Specifically, the years of experience of a useridentified by a trainer ID “T001” and a name “Tanaka” is “5 years”, andthe years of experience of a user identified by a trainer ID “T003” anda name “Kato” is “10 years”.

Therefore, in a situation, when the user (trainee) who receives trainingselects a user (trainer) who instructs, for example, the user (trainee)can select a user who instructs with reference to the years ofexperience and therefore can receive training with more peace of mind.By being selected as the user who instructs based on the information,the user (trainer) who instructs can raise his or her name recognition.

The item “specialty” is information indicating a specialty of trainingof the user (trainer) who instructs the user (trainee) who receivestraining. Specifically, the user identified by the trainer ID “T001” andthe name “Tanaka” has specialties “yoga” and “exercise”, and the useridentified by the trainer ID “T002” and the name “Saito” has specialties“strength training” and “exercise”.

The server 20 or the like may determine the user (trainer) who instructsthe user who receives training based on the information of the item“purpose” and the item “ideal figure” in the trainee informationdatabase 2601 for the user who receives training. In this way, it ispossible to make a match between a trainee and a trainer appropriate tothe purpose of training or the like of the trainee can be made. Forexample, the server 20 or the like can allocate, to a trainee whosepurpose is exercise, a trainer who is good at a training appropriate tothe purpose as an instructor.

The item “required physical fitness level” indicates a physical fitnesslevel defined for each user (trainer) who instructs a user (trainee) whoreceives training that is required for receiving training from the user(trainer). Specifically, the required physical fitness level of the user(trainer) identified by the trainer ID “T001” and the name “Tanaka” is“3 or higher”. The physical fitness level of the user (trainee)identified by the user ID “U001” and the name “Mori” described above is“5”, so that the user (trainee) has achieved the physical fitness levelrequired for receiving instruction from the user (trainer). On the otherhand, the physical fitness level of the user (trainee) identified by theuser ID “U002” and the name “Sato” is “2”, so that the user (trainee)has not achieved the physical fitness level required for receivinginstruction from the user (trainer).

In this way, the user who receives training can receive trainingappropriate to the physical fitness level of the user. The user whoinstructs the user who receive training can give training to users of aprescribed level or higher, so that variations of training regimens orachievement levels among the users (trainees) can be reduced, andtherefore, the users who receive training can be kept motivated. Theserver 20 or the like may make a match between a trainer and a traineebased on the item “required physical fitness level” for the trainer andthe item “physical fitness level” for the trainee.

The item “lesson type” indicates a type of lesson that can be given bythe user (trainer) who instructs the user who receive training.Specifically, the lesson type of the user identified by the trainer ID“T001” and the name “Tanaka” is “private” or “group”, and the lessontype of the user identified by the trainer ID “T002” and the name“Saito” is only “private”.

This item shows whether the user (trainer) who instructs a user whoreceive training is suitable for private lessons or group lessons.Therefore, when making a match between a trainer and trainee, the server20 or the like can sort trainers into one-to-one private lessons andone-to-many group lessons based on the “lesson type”. Furthermore, theterminal apparatus 10 of the trainee may presents the “lesson type” ofthe trainer to the trainee.

The item “remarks” is information on special notes or the like on theuser (trainer) who instructs the user who receive training. The specialnotes include information on objective assessment of achievements of thetrainer as a training instructor. Specifically, the user identified asthe trainer ID “T001” and the name “Tanaka” is a “XXX Grand Prixwinner”.

Therefore, in a situation, when the user (trainee) who receives trainingselects a user (trainer) who instructs the user who receive training, ifthe information of the item “remarks” is presented on the terminalapparatus 10 of the trainee to the trainee, the trainee can select atrainer by taking the information into consideration. By entering theinformation, the user (trainer) who instructs the user who receivestraining can expect to increase the possibility of being selected as aninstructor.

The training history database 2603 includes an item “training ID”, anitem “date and time”, an item “meeting ID”, an item “trainee ID”, anitem “trainer ID”, an item “sensing result”, an item “details ofinstructions”, an item “assessment of trainer”, and an item“match-making ID”, for example.

The item “training ID” is information for identifying a traininginvolving a trainer and a trainee.

The item “date and time” indicates the date and time of the training.

The item “meeting ID” is information for identifying a communicationsession in a case where the terminal apparatus 10B of the trainer andthe terminal apparatus 10A of the trainee communicate with each otherfor an online training lesson. For example, information for identifyinga communication session includes a URL or the like for an onlinemeeting.

The item “trainee ID” is information for identifying the trainer whogives training.

The item “trainer ID” is information for identifying the trainee whoreceives training instructions from the trainer.

The item “sensing result” indicates a result of measurement (point clouddata) of the movement of the body of the trainer during training by theterminal apparatus 10A with the measurement unit 30 or a video obtainingby rendering the point cloud data in a virtual space based on settingsof a virtual camera (the video includes an avatar representing themovement of the body of the trainee in the form of a set of graphicelements based on point cloud data and an auxiliary line drawn accordingto the way of movement of the body of the trainee (that is, the type ofmovement of the body presented to the user, such as lateral armelevation)), for example.

The item “details of instructions” indicates details of instructionsconcerning how to move the body of the trainee given by the trainer tothe trainee. For example, when the trainer tells the trainee how to movehis or her body (“How to move his or her body” includes advice to guidethe user into a desired posture, such as advice of the trainer to guidethe user into a desired posture when a part (arm or leg) of the body isbent compared with the desired posture), the voice including theinstruction or a result of analysis of the voice is retained in the item“details of instructions”.

The item “assessment of trainer” indicates an assessment of the trainerby the trainee. For example, after an online training, the terminalapparatus 10A of the trainee presents a screen for inputting assessmentof the trainer to the trainee, and receives assessment of the trainer bythe trainee.

The item “match-making ID” is information for identifying a matchbetween a trainer and a trainee made by the server 20. In response to amatch being made, the trainer can start training of the trainee.However, no training may start even if a match is made.

Example Operation in Fifth Embodiment

In the following, there will be described a series of various kinds ofprocessing performed when the first processor of the terminal of thefirst user (trainee) of the system 1 senses the movement of the body ofthe first user as point cloud data, and the first processor transmitsinformation based on the point cloud data on the first user to theterminal of the second user (trainer) in the fifth embodiment.

(Sensing of Movement of First User and Transmission to Terminal ofSecond User)

FIG. 27 is a flowchart showing a series of processing in which themovement of the body of the first user is sensed as point cloud data,and information based on the point cloud data on the first user obtainedby the sensing is transmitted to the terminal of the second user.

Concerning the terminal apparatus 10 shown in FIG. 1, in the followingdescription, the terminal apparatus 10 of the first user will bereferred to as the “terminal apparatus 10A”, and a component of theterminal apparatus 10A will be referred to as a “processor 19A” or thelike. Similarly, the terminal apparatus 10 of the second user will bereferred to as the “terminal apparatus 10B”, and a component of theterminal apparatus 10B will be referred to as a “processor 19B” or thelike.

In the example in FIG. 27, the terminal apparatus 10A of the first userand the terminal apparatus 10B of the second user transmit and receivevideo taken with the camera 160 of each terminal and audio picked up bythe microphone 141 of each terminal to and from one another byestablishing communication therebetween.

For example, the terminal apparatus 10A and the terminal apparatus 10Bexecute an application for online video meeting, and establishcommunication based on identification information for participating inthe meeting (such as an ID, a URL or the like for the meeting issued bythe application).

The terminal apparatus 10A obtains point cloud data on the movement ofthe body of the first user from the measurement unit 30 or the like. Theterminal apparatus 10A puts the point cloud data in a virtual space,performs rendering with virtual cameras arranged in the virtual space ata front position with respect to the body of the first user (along theanteroposterior axis in the description of FIG. 11) and a side positionwith respect to the body of the first user (along the transverse axis inthe description of FIG. 11), and displays the rendering result. Therendering result includes an avatar in the form of the set of graphicelements based on point cloud data representing the first user viewedfrom the front and an avatar in the form of the set of graphic elementsbased on point cloud data representing the first user viewed from theside. The terminal apparatus 10A draws an auxiliary line correspondingto the way of movement of the body of the user in association with aregion in which the rendering result (the avatar viewed from the frontand the avatar viewed from the side) described above is displayed.

The terminal apparatus 10A displays an image of the avatars and thedrawn auxiliary lines described above on the screen, and shares thescreen with the terminal apparatus 10B through an online video meetingapplication. In this way, the video of the first user taken by a camera160A, the audio picked up by the microphone 141, and the image of theavatars and the drawn auxiliary line are transmitted from the terminalapparatus 10A to the terminal apparatus 10B. The second user of theterminal apparatus 10B checks the movement of the body of the first userby watching the video taken by the camera 160A and the image (the imageof the avatars and the drawn auxiliary line) on the shared screen, andnotifies the first user of an audio or other advice to the first user.

In step S2701, the processor 19A of the terminal apparatus 10A of thefirst user senses the movement of the body of the first user as pointcloud data by means of the measurement unit 30 or the like.

Specifically, the processor 19A displays a regimen for body movements tobe performed by the first user to the first user to prompt the firstuser to make a particular movement (such as a movement of lateral armelevation or a movement of a training item (such as squats)) for bodymovement sensing. The processor 19A of the terminal apparatus 10A of thefirst user takes an image of the particular movement by means of themeasurement unit 30 or the like, thereby sensing the movement of thebody of the first user as point cloud data.

In step S2703, the processor 19A generates video including the avatarrepresenting the first user (trainee) and the auxiliary linecorresponding to the type of the movement performed by the first user(the type of the exercise regimen presented to the first user) asinformation based on the point cloud data on the first user obtained bysensing by the measurement unit 30 or the like.

In step S2705, the processor 19A displays the video generated in stepS2703 on the display 132A of the terminal apparatus 10A. The processor19A transmits the displayed video (the video including the avatar andthe auxiliary line) and a taken image (which is not a set of graphicelements based on point cloud data but an image taken by an RGB camera)generated by the camera 160A taking an image of the first user to theterminal apparatus 10B of the second user.

In step S2751, the processor 19B of the terminal apparatus 10B of thesecond user receives the video generated by the terminal apparatus 10A(which includes the avatar and the auxiliary line) as information basedon the point cloud data on the first user from the terminal apparatus10A of the first user. The processor 19B receives the taken image of thefirst user (the taken image that is not a set of graphic elements basedon point cloud data) generated in step S2705.

In step S2753, the processor 19B displays, on the display 132B or thelike, video including a first avatar in the form of a set of graphicelements based on point cloud data corresponding to the movement of thebody of the first user based on the video (which includes the avatar andthe auxiliary line) and the taken image received in step S2751. Theprocessor 19B displays the taken image of the first user (which is not aset of graphic elements based on point cloud data) on the display 132Bor the like. In this way, the first avatar and the taken image of thefirst user are presented to the second user as information on themovement of the body of the first user.

In step S2703, the processor 19A may determine various kinds ofinformation on the movement of the first user (such as that what part ofthe body is easy to move or what part of the body is difficult to move),the physical fitness level of the first user or the like based on thetrainee information database 2601 or the like of the server 20. Theprocessor 19A may store the determined information in the storage unit180 of the terminal apparatus 10A of the first user, or may transmit thedetermined information to the server 20 so that the information isstored in the trainee information database 2601 of the storage unit 202of the server 20.

The processor 19A may determine the second user (trainer) to whom theinformation is transmitted by referring to the trainer informationdatabase 2602 based on the result of sensing of the movement of the bodyof the first user by the measurement unit 30 or the like.

In this way, the user as a trainee can receive training from the optimalinstructor based on the information on the movement of his or her bodyat the start of the training.

In step S2751, the terminal apparatus 10B may receive data of sensing bythe measurement unit 30 or the like from the terminal apparatus 10A ofthe first user, and perform rendering based on the point cloud data inthe received information. That is, the terminal apparatus 10B, ratherthan the terminal apparatus 10A, may render the first avatarrepresenting the movement of the body of the first user based on thepoint cloud data measured by the terminal apparatus 10A. The terminalapparatus 10B generates video including the first avatar, and displaysthe generated video on the display 132B or the like. In this processing,the movement of the body of the first user may be drawn from a pluralityof points of view, and video including the first avatars drawn from thepoints of view arranged side by side may be presented to the seconduser.

In a situation, the terminal apparatus 10B may draw an auxiliary line ata position determined based on a part of the body used for evaluatingwhether the movement of the body of the first user is ideal or not, andvideo including the drawn auxiliary line to the second user. Forexample, when the trainer evaluates whether the movement of the body ofthe user is ideal or not based on whether the user's back isstraightened or not or whether the user's knee is at a predeterminedangle with respect to the ground in the exercise being performed by theuser, the terminal apparatus 10B displays an auxiliary linecorresponding to the shape of the user's body with his or her backstraightened in the vicinity of the spine part of the avatar. Whenevaluating whether the knee is at a predetermined angle with respect tothe ground or not, the terminal apparatus 10B displays an auxiliary linebent at the predetermined angle in the vicinity of the knee part of theavatar (or in a region in which the knee part is displayed).

Furthermore, the way of displaying a surface part other than the jointsof the body may be modified based on the part of the body used forevaluating whether the movement of the body of the first user is idealor not, and video modified in the way of displaying a body surface maybe presented to the second user.

A surface part other than the joints of the body is a surface recognizedas the appearance of the body and is the surface of the abdomen, forexample. For example, the terminal apparatus 10B may display differentcolors for different cases where the movement of the body of the firstuser is too big compared with the ideal movement and where the movementof the body of the first user is too small compared with the idealmovement. For example, the abdomen is displayed in a warm color when theabdomen is positioned too forward compared with the ideal movement, andis displayed in a cold color when the movement of the abdomen is toosmall compared with the ideal movement. In this way, the first user orthe second user can easily see how the first user should move his or herbody in order that the first user can move his or her body in an idealway. In addition, the user can train by grasping the degree of deviationof the movement of the body of the user from the ideal movement. Inaddition, since the way of displaying the surface part of the body ismodified, in addition to the movements of joints, movements of muscleson the body surface can also be grasped.

In the example described above, concerning the video including the firstavatar of the first user displayed on the display 132B or the like, boththe image of the first user taken when generating the first avatar andthe video of the first avatar are presented to the second user.Alternatively, the terminal apparatus 10B of the trainer may refer tothe trainee information database 2601 and display a screen on thedisplay 132 or the like in accordance with the specification in the item“display on screen”.

For example, the terminal apparatus 10B may display the movement of thebody of the first user as the first avatar (that is, a set of graphicelements based on point cloud data) but may not display the taken imageof the first user (which is not a set of graphic elements based on pointcloud data). When a plurality of trainees receive training from atrainer, the taken image of each trainee (which is not a set of graphicelements based on point cloud data) may be able to be displayed on theterminal apparatus 10 of the trainer but may not be displayed on theterminal apparatuses 10 of the other trainees who are receiving trainingfrom the trainer at the same time. In this way, it is possible that,while both the point cloud data and the taken image of each trainee aredisclosed as the sensing result to the trainer, the trainees do notshare their taken images, although sharing their avatars (sets ofgraphic elements based on point cloud data).

In this way, the user can train while keeping his or her privacy.

In another situation, at least any of the processor 19A, the processor19B, and the processor 29 of the server 20 may define a virtual space,and arrange a first object for displaying the video including the firstavatar associated with the first user and a second object for displayingvideo including a second avatar associated with a third user who trainstogether with the first user in the virtual space, and the processor 19Aor the processor 19B may display video including the first object andthe second object in the virtual space on the display 132A or thedisplay 132B or the like.

In this way, when a plurality of users trains together, the users canexperience the training in the same virtual space and feel a sense ofunity and a sense of accomplishment in the training. Therefore, themotivation of the users can be maintained or improved.

Next, with reference to FIG. 28, a flow of a process in which a trainerinstructs a trainee online will be described.

In the following description, an example will be described in which thetrainer instructs the trainee with a voice and controls an applicationrunning on the terminal apparatus 10 of the trainee with a voice of thetrainer, so that the trainer can easily obtain information useful forinstruction. Specifically, on the terminal of the trainee, the trainercan control, through voice input, drawing of an avatar based on pointcloud data, drawing of an auxiliary line, determination of the type ofthe movement of the body to be assigned to the trainee and the like.

FIG. 28 is a flowchart showing a flow of a process in which the terminalof the trainer receives a voice input from the trainer and transmits thevoice input to the terminal of the trainee.

In step S2851, the processor 19B of the terminal apparatus 10B of thesecond user receives a voice input from the second user through themicrophone 141B or the like, and transmits the voice of the second userfrom the terminal apparatus 10B of the second user to the terminalapparatus 10A of the first user.

In step S2801, the processor 19A of the terminal apparatus 10A of thefirst user outputs the voice of the second user received from theterminal apparatus 10B to the first user through the speaker 142A or thelike.

In step S2803, the processor 19A analyzes the voice of the second userreceived from the terminal apparatus 10B by voice recognition. Theprocessor 19A determines contents of the voice of the second user basedon the result of the analysis by voice recognition. The processor 19Adetermines an instruction to the application that draws an avatar basedon the point cloud data on the first user based on the determinedcontents of the voice of the second user. The processor 19A controls theapplication based on the determined instruction to the application.

Specifically, the contents of the voice of the second user determined bythe processor 19A include

(i) a content for bringing the movement of the body of the first userclose to an ideal movement. For example, the processor 19A recognizes aword relating to a part (such as knee, elbow, arm, or shoulder) of thebody of the user in the voice of the second user through voicerecognition. The processor 19A further recognizes a word (such as “pullyour knee toward your chest”, “keep your leg horizontal”, or “straightenyour back”) indicating a movement that brings a part of the body of thefirst user close to an ideal movement in the voice of the second user.

(ii) The contents of the voice determined by the processor 19A include acontent indicating a type of movement (such as a training regimen)assigned to the first user for training of the first user.

When the contents of the voice of the second user include a type ofmovement, the processor 19A sets the type of movement to be assigned tothe first user to be the movement specified by the voice of the seconduser, and selects an auxiliary line associated with the specified typeof movement as the auxiliary line associated with the avatarcorresponding to the first user. In this way, the second user can moreeasily evaluate the movement of the body of the first user.

(iii) The contents of the voice determined by the processor 19A includea content that makes the application running on the terminal apparatus10A that draws the avatar of the first user and the auxiliary lineperform particular processing. For example, commands to the applicationindicated by the voice of the second user include the following.

-   -   A command concerning the way of drawing of the avatar (which        includes the position of the virtual camera arranged with        respect to the point cloud data representing the movement of the        body of the first user. That is, the way of drawing of the        avatar includes whether the avatar corresponding to the first        user is drawn from the front, from the side, or from another        direction, for example).    -   A command concerning the way of drawing of the auxiliary line        (which includes drawing the auxiliary line parallel to the        movement of the body of the user, drawing the auxiliary line        along the shape of the body of the user, or whether to draw an        auxiliary line or not, for example).    -   A command concerning the way of drawing of the avatar in a case        where there is a deviation from the ideal movement (for example,        a particular part of the body is drawn in different colors when        the movement of the particular part is too big or too small        compared with the ideal movement).    -   A command to suspend the application or other commands.

When the processor 19A recognizes the speaker as the trainer throughvoice recognition (when the speaker is a trainer registered in thetrainer information database 2602, for example), the processor 19A maycontrol the application drawing the avatar of the first user based onthe voice of the speaker. In this way, the application can be preventedfrom being accidentally controlled by the other users than the trainer.When the processor 19A communicates with another terminal apparatus,such as the terminal apparatus 10B, through a web conferencing tool orthe like, the processor 19A may control the application drawing theavatar of the first user based on information on the speaker identifiedby the web conferencing tool or the like. For example, the webconferencing tool or the like may display information (such as icons)for identifying the user participating in the meeting, and display theinformation for identifying the user who is speaking so as to bedistinguishable from the information on the other users. When the webconferencing tool or the like is displaying the icon or the likecorresponding to the trainer so as to be distinguishable from the iconsof the other users, the processor 19A may perform the processingdescribed above by regarding the trainer as the speaker.

For example, when a plurality of trainee receives training from thetrainer, depending on the voices of the trainees, the application may beinhibited from performing processing relating to the training (such asprocessing of changing the type of the movement assigned to the traineesor processing of changing the auxiliary line to be drawn along with theavatar in response to the change of the type of the movement), and thetrainer may perform the processing. Then, even if the users speak duringtraining, the users can train as intended by the trainer.

As described above, the second user can change the way of display of thevideo including the first avatar by instructing the first user with avoice, and can continue instructing the first user based on the changedvideo including the avatar.

(Sensing of the Movement of the First User and Identification of anInstruction Based on a Learned Model)

FIG. 29 is a flowchart showing a process in which the movement of thebody of the first user is sensed as point cloud data, and an instructionto the first user is identified based on information on the point clouddata on the first user obtained by the sensing and a learned model.Through this process, even if no trainer is present, the trainee cancheck the movement of his or her body with the learned model.

In step S2901, the processor 19A of the terminal apparatus 10A of thefirst user senses the movement of the body of the first user as pointcloud data by means of the measurement unit 30 or the like.

In step S2902, the processor 19A determines details of instruction tothe first user based on information on the point cloud data on the firstuser and a learned model.

Here, methods of creating a learned model will be described.

(i) For example, information on point cloud data on one or more users isobtained, and then a learned model is created by associating theinformation on the point cloud data on the one or more users withdetails of training based on various kinds of information (such as age,weight and purpose), information on the user (trainer) who instructs theuser(s) receiving training, and the details of training actuallyreceived by the trainee(s), and information on the history of change ofthe information on the trainee(s) caused by the training, for example.

That is, it can be said that the learned model is a model that learnsinformation on the trainee and details of instruction by the trainer astime-series data and outputs how much the trainee will be changed bytraining.

The learned model may associate the information based on the point clouddata, which is training data for the learned model, with obtainedinformation on the user (that is, information on the trainee) orinformation on the trainer who has instructed the trainee. In this way,when selecting a learned model for the trainee, the trainee or trainercan refer to the information on the trainee or the information on thetrainer associated with the learned model. Alternatively, the processor19A or the like may determine the learned model to be proposed to theuser as a trainee based on the user information associated with thelearned model.

(ii) Alternatively, the learned model may be created using, as trainingdata, time-series data of results of measurement of the movement of thebody of the trainee (such as point cloud data and a taken image of thetrainee) by the trainer tagging the time-series data (such as a commentconcerning the way of moving a part of the body in an ideal manner, theimportance of breathing, or that the heartrate is too high and a breakis needed).

For example, in a situation, based on the measurement result of themovement of the body of the trainee and the learned model, a trainingregimen appropriate to the constitution of the user as a trainee, thetiming of breathing in the training, the timing when the heartraterises, the timing of hydration, or the timing of a break can bedetermined and presented to the user, for example.

For example, in a situation, when the trainee wears a wearable device(such as the watch type, the glasses type, or the ring type) and usesthe learned model while the wearable device measures the heartrate, thetiming of breathing or the like of the trainee user, the learned modelmay output details of instruction to be given to the trainee.

In another situation, the exercise intensity (severity of exercise)including calorie consumption may be determined from informationconcerning the point cloud data on the user to present advice abouttaking a break or hydration to the user.

(iii) In a method of creating a learned model, a learned model may becreated using, as teacher data, the measurement data (measurementhistory database 283) accumulated as a result of the ROM check describedin the first embodiment. For example, a learned model may be createdbased on the measurement data, a training regimen performed by the user,and a response to a questionnaire or the like input by the user.

In this way, the learned model created based on the measurement data ofthe ROM check may output the type of exercise to be proposed to theuser, details of advice to the user, or the like.

In a method of creating a learned model, a learned model may be createdby using, as teacher data, the measurement data (measurement resultdatabase 287) accumulated as a result of evaluation of the walkingposture of the user described in the second embodiment. For example, alearned model may be created based on the measurement data, a trainingregimen performed by the user, and a response to a questionnaire or thelike input by the user.

In this way, the learned model created based on the measurement data forwalking posture evaluation may output the type of exercise to beproposed to the user, details of advice to the user, or the like.

In a method of creating a learned model, a learned model may be createdbased on measurement data accumulated as a result of estimation of theposture of the user (such as joint positions). Specifically, in a methodof the server 20 or the like creating a learned model, a learned modelis created based on the result of estimation of the positions of jointsof the user, a training regimen performed by the user, and details ofadvice from the trainer, and the created learned model may be stored.

In this way, the server 20 may determine details of instruction to theuser based on information based on the estimation result of the postureof the user and the learned model, and present the determined details ofinstruction to the user.

Furthermore, the server 20 or the like may store the learned model inassociation with information on the user whose data has been used astraining data for the learned model (the result of estimation of theposture of the user).

Furthermore, the server 20 or the like may determine the exerciseintensity of the user in the same manner as described above based oninformation based on the estimation result of the posture of the userand the learned model. The server 20 may determine details ofinstruction to the user based on the determination result of theexercise intensity of the user, and present the determined details ofinstruction to the user.

In step S2903, the processor 19A presents the details of instructiondetermined by the learned model to the first user by displaying thedetails of instruction on the display 132A or the like, for example.

Example Screens in Fifth Embodiment

FIG. 30 to FIG. 32 are diagrams showing example screens displayed on theterminal apparatus 10.

FIG. 30 shows an example screen when the terminal apparatus 10A of thefirst user senses the movement of the body of the first user as pointcloud data and transmits information based on the point cloud data onthe first user obtained by the sensing to the terminal of the seconduser.

In FIG. 30, a camera 2901 of the measurement unit 30 is an apparatus forsensing the movement of the body of the first user.

Specifically, the camera 2901 takes an image of the first user throughan image-taking part of the camera 2901. The camera 2901 is configuredto be capable of measuring distance and direction, and is an RGB-Dcamera or a camera based on the laser imaging detection and ranging(LIDAR) technology, for example.

A last-time window 2902 is a window for displaying an image includingthe avatar (that is, an image drawn based on point cloud data) of thefirst user in the last training of the same type when the first userperform training. For example, the terminal apparatus 10A obtains thepoint cloud data representing the movement of the body of the user inthe past training of the first user by referring to the training historydatabase 2603 of the server 20.

In the example shown, in the last-time window 2902, the avatar of thefirst user is displayed as a set of figures, such as circles. However,the avatar of the first user may be displayed as a set of graphicelements based on point cloud data measured by the measurement unit 30.

By displaying an avatar based on point cloud data from past training ofthe first user, the first user and the second user can easily check howmuch the movement of the body of the first user has been changed bytraining. For example, when the first user makes a movement of curlingup his or her body holding his or her knees to his or her chest, theterminal apparatus 10A displays the last-time window 2902, which is aprevious-training window. The first user or the second user can comparewhat is displayed in a this-time window 2903 described later with thelast-time window 2902. Therefore, the first user can check a change ofthe movement of his or her body.

The terminal apparatus 10A may constantly display the last-time window2902 or may turn on and off the display of the window in response to aninstruction from the first user or the second user. As the avatar of thefirst user, the terminal apparatus 10A may display images of the firstuser drawn from a plurality of viewpoints, such as front, side, orabove. Furthermore, an auxiliary line can also be drawn at a positionbased on a part of the body used for evaluating whether the movement ofthe body of the first user is ideal or not. In the example shown, in thelast-time window 2902, auxiliary lines are drawn in association with theregion where the avatar of the first user is displayed.

The this-time window 2903 is a window for displaying an image of theavatar of the first user based on the result of sensing by themeasurement unit 30 when the first user is performing training.

In the example shown, a first user 2094 is displayed, and the first useris actually performing training on a mat or the like. The terminalapparatus 10A takes an image of the movement of the first user by meansof the camera 2901 to obtain information on point cloud data on thefirst user.

A first user image 2905 is an image of the first user taken by thecamera 160 or the like. In the first user image 2905, the first user isnot drawn as a set of graphic elements based on point cloud data but asan image taken by a 2D camera or the like. That is, in the first userimage 2905, a taken image equivalent to what would be seen in person isdisplayed.

In this way, the terminal apparatus 10A allows the first user to comparethe actual image of the first user with an image based on information onpoint cloud data. Furthermore, in response to specification by the firstuser, the terminal apparatus 10 may display an icon, a character stringor the like representing the first user that can be identified by thesecond user or the like while displaying no taken image of the firstuser as the first user image 2905.

A second user image 2906 is an image of the second user who instructsthe user who receive training. The terminal apparatus 10A receives ataken image of the second user from the terminal apparatus 10B of thesecond user, and displays the taken image of the second user as a seconduser image 2906. In this way, in the training, the first user and thesecond user can communicate with each other by seeing each other's imagethrough video call between the terminals via a network 80 or the like.

FIG. 31 shows example screens for the first user and the second user andan overall arrangement of the system when the movement of the body ofthe first user is sensed as point cloud data, and information based onthe point cloud data on the first user obtained by the sensing istransmitted to the terminal of the second user.

In FIG. 31, a camera 3001 is an apparatus for sensing the movement ofthe body of the first user as point cloud data, as with the camera 2901in FIG. 30.

This-time windows 3002 and 3006 are windows for displaying an image ofthe first user as a set of graphic elements based on point cloud datawhen the first user is performing training, as with the this-time window2903 in FIG. 30. The this-time window 3002 is displayed on the display132A or the like of the terminal apparatus 10A of the first user, andthe this-time window 3006 is displayed on the display 132B or the likeof the terminal apparatus 10B of the second user.

A first user 3003 is shown as being actually performing training on amat or the like, as with the first user 2904 in FIG. 30.

First user images 3004 and 3007 are not video including an avatar butactual images of the first user taken by the camera 2901, as with thefirst user image 2905 in FIG. 30. The first user image 3004 is displayedon the display 132A or the like of the terminal apparatus 10A of thefirst user, and the first user image 3007 is displayed on the display132B or the like of the terminal apparatus 10B of the second user.

Second user images 3005 and 3008 are images of the second user whoinstructs the user who receives training, as with the second user image2906 in FIG. 30. The second user image 3005 is displayed on the display132A or the like of the terminal apparatus 10A of the first user, andthe second user image 3008 is displayed on the display 132B or the likeof the terminal apparatus 10B of the second user. In a situation, asshown in this example screen, in response to a voice input operation bythe second user, the speaker 142A or the like of the terminal apparatus10A may output the voice of the second user (such as a voice “Lower yourback down!”).

The second user 3009 is instructing based on the image of the first userby watching the image displayed on the display 132B or the like of theterminal apparatus 10B. The second user can instruct the first user witha voice. The second user can advise the first user about the movement ofthe body by checking, as the taken image of the first user, one or bothof an image based on point cloud data and an image equivalent to whatwould be seen in person. The set of graphic elements based on pointcloud data is reduced in the amount of information given to the seconduser compared with the image of the first user actually taken, so thatthe second user can more easily see the movement of the body of thefirst user and more easily evaluate the movement and give advice.

FIG. 32 shows an example screen when the movements of a plurality ofusers are sensed as point cloud data, and information based on the pointcloud data on the users obtained by the sensing is transmitted to adefined virtual space.

In FIG. 32, a virtual space 3101 is a space in which one or more userswho are receiving training can each put an object for displaying videoincluding an avatar associated with himself or herself.

A trainer image 3102 is an image of the second user who instructs a userwho receives training. The terminal apparatus 10B of the second usertakes an image of the second user, and transmits the image to a computerthat controls the virtual space as required.

A user image region 3103 is a region in which images of the one or moreusers performing training are displayed. The image displayed in theregion may be only video including an avatar in the form of a set ofgraphic elements based on point cloud data associated with the user or acombination of the video including an avatar and an image of the useractually taken that is different from the video including an avatar.

In this way, each user can put video including an avatar associated withthe movement of the body of the user in the virtual space. Since eachuser can see the images of the avatars of the users put in the virtualspace, each user can feel as if the user is performing training in thesame place as the other users. This can allow each user to feel a senseof unity and a sense of accomplishment in the training, and can motivateeach user to continue training. In response to a match being madebetween a trainer and trainees who will receive training, the server 20may enable the terminal of the trainer and the terminals of the traineesto communicate with each other, and build a virtual space in any of theserver 20 or the terminal of each user, for example. That is, thecomputer serving as a host for the communication between the users maybe specified by the server 20.

6. Example Modifications

Example modifications of this embodiment will be described. That is, thefollowing implementations are possible.

(1) In the examples in FIG. 31 to FIG. 32 described above, examples havebeen described in which an avatar based on the result of sensing of auser when the user received training in the past and an avatar based onthe result of sensing of the user currently receiving training from atrainer are displayed side by side.

In addition, an avatar of a trainee may be displayed side by side withthe avatars based on time-series point cloud data on the traineeobtained by sensing over time during the training with the trainer.

That is, an avatar based on point cloud data obtained by sensing at thetiming of start of the training or within a certain time period from thestart of the training and an avatar based on the result of real-timesensing in the training under the instruction of the trainer or theresult of sensing at the timing when a certain time has elapsed sincethe start of the instruction by the trainer may be displayed side byside by the terminal apparatus 10A.

In one session of training, if the movement of a part of the body of thetrainee has come close to an ideal movement, it may be reflected in thedisplayed avatar by highlighting the part compared with the other parts.For example, the processor 19A may evaluate how much the movement of thebody of the trainee has come close to the auxiliary line drawn alongwith the avatar, and make a drawing on the screen based on theevaluation result. For example, if the movement of straightening thetrainee's knee in a particular direction has come close to an idealmovement, the processor 19A draws an auxiliary line extending in theparticular direction. The processor 19A may change the way of drawing ofthe knee part of the avatar or the auxiliary line associated with theknee part as the movement of the knee comes close to the auxiliary linewhen the trainee is moving his or her body.

As a result, the trainee can easily check how much the trainee after onesession of training is different from the trainee before the training orhow much the trainee has changed in the course of the training.Therefore, the trainee can easily check how much the trainee has changed(for example, the trainee has come close to his or her goal). Therefore,the motivation of the trainee to continue training can be furtherimproved.

In the case of online training, the amount of visual information issmaller than in the case of in-person training (since the trainer cansee the trainee with his or her own eyes in the case of in-persontraining, whereas the trainer see a two-dimensional image on the displayin the case of online training), so that the quality of communicationand experience may be low. In the case of online training, in addition,it is extremely difficult for the trainer to touch the body of thetrainee. Therefore, by feeding back the result of training to thetrainee as described above, the trainee can be motivated to continuetraining online.

When the movement of the body of the user has come close to an idealmovement in one session of training as described above, a learned modelmay be created based on the sensing result.

(2) In the embodiments described above, examples have been described inwhich an avatar is displayed based on the result of sensing of themovement of the body of a user, and an avatar corresponding to thetrainer is displayed to the trainee in online training between thetrainer and the trainee.

As an alternative, the trainer and the trainee may not communicate witheach other, and the user may train by checking the movement of his orher body through the avatar described above for himself or herself. Forexample, on the terminal apparatus 10A, the user may set a way ofmovement of the body, draw an auxiliary line or the like in accordancewith the set way of movement of the body, and notify the user of the wayof movement of the body (or provide the user with a guide how to movehis or her body), and the measurement unit 30 or the like may sense themovement of the body of the user.

(2-1) A Method of Sensing the Movement of the Body of the User

The terminal apparatus 10A presents, to the user, the result of sensingof the movement of the body of the user in the form of an avatar or thelike. As described in the first and second embodiments, for example, theterminal apparatus 10A can determine the positions of joints of the bodyof the user and estimate the skeleton (a set of bones) (FIG. 12).

The terminal apparatus 10A may estimate the posture of the user based onthe result of estimation of the positions of joints and the skeleton ofthe user as described below.

-   -   Based on a result of imaging of the user as a subject (a        two-dimensional movie), the terminal apparatus 10A estimates the        posture of the user in the movie without point cloud data.

That is, the terminal apparatus 10A may take an image of the user bymeans of a camera that is not configured to optically measure distanceto the subject unlike LiDAR or the like, and estimate the positions ofjoints, the skeleton or the like as the posture of the user in the takenimage based on the result of the imaging.

For example, by preparing a learned model that receives the taken imageas an input and outputs the posture of the person who is the subject ofthe taken image, such as the positions of joints or the skeleton of theperson, the terminal apparatus 10A may be able to estimate the postureof the user based on the result of imaging of the user and the learnedmodel.

Such a learned model may be created by preparing training data includinginformation on the positions of joints, the skeleton or the like in thetaken image of the person who is the subject of the taken image andtraining a computer with the training data.

-   -   As described in other embodiments, the terminal apparatus 10A        senses, as point cloud data, the movement of the body of the        user by means of a camera installed in the terminal apparatus        10A or the like or the measurement unit 30 or the like, and        estimates the positions of joints, the skeleton or the like        based on the result of the sensing.

(2-2) Drawing of an Auxiliary Line Based on the Result of Estimation ofthe Positions of Joints and the Skeleton

The terminal apparatus 10A may draw an auxiliary line serving as areference for the user to move his or her body based on the result ofestimation of the posture of the user. The terminal apparatus 10A maydraw the auxiliary line based on the result of estimation of the postureof the user as follows.

-   -   The terminal apparatus 10A or the like determines the position        of an auxiliary line to be drawn based on the type of movement        of the body to be presented to the user (such as a movement for        evaluating the movement of trunk muscles or the joint range of        motion, such as lateral arm elevation, or a movement for        strength training), the positions of joints, and the result of        estimation of the skeleton.

For example, some types of movement of the body include a movement thatinvolves a certain amount of movement of the body of the user (such as amovement of stepping a foot forward for evaluation of the joint range ofmotion). The terminal apparatus 10A displays an avatar to the user, andat the same time draws an auxiliary line serving as a guide for theamount of body movement based on the result of estimation of theposition of a particular joint of the user.

For example, when the user makes a movement of stepping a foot forward,the terminal apparatus 10A first prompts the user to assume apredetermined posture (for example, to assume an upright posture beforestepping a foot forward) through audio, video or the like. Based on thepositions of joints (such as the shoulder joints, the hip joints, or theknee joints) in this posture, the terminal apparatus 10A estimates thestep length of the user (for example, the required step length of theuser can be determined based on the height and the leg length of theuser), and determines the position of the auxiliary line to be drawnbased on the estimated step length and the result of estimation of thepositions of joints.

For each type of movement of the body of the user, the terminalapparatus 10A may store the positions of joints to be referred to whendrawing an auxiliary line and the position of the auxiliary line drawnwith respect to the positions of joints associated with each other inadvance.

FIG. 33 is a diagram showing a concept of a process in which theterminal apparatus 10A draws an auxiliary line based on the result ofestimation of the posture of the user.

In the example (A) in FIG. 33, the terminal apparatus 10A senses themovement of the body of the user as point cloud data. Based on theresult of sensing, the terminal apparatus 10A stores three-dimensionalobject data on an avatar AVT_A corresponding to the user. Based on thethree-dimensional object data on the avatar AVT_A, the terminalapparatus 10A estimates the positions of joints (such as the jointsshown in FIG. 12) of the user represented by the three-dimensionalobject data. In the example (A), the terminal apparatus 10A determines ajoint position JNT_01A, the coordinates of which are the result ofestimation of the position of the hips joint of the user represented bythe avatar AVT_A when the three-dimensional object data on the avatar isput in a virtual space.

In the example (A) in FIG. 33, in accordance with the type of movementof the body of the user, the terminal apparatus 10A determines toarrange three-dimensional object data on an auxiliary line LN_01A at aposition at a distance of a certain distance DIST_01A from the jointposition JNT_01A. The terminal apparatus 10A may set the certaindistance based on the physique of the user (which is obtained bymeasuring the height, the siting height, the shoulder width or the likebased on the result of sensing of the movement of the body of the user).The reference amount of movement of the body of the user can bedetermined by the physique of the user. In the example (A), the terminalapparatus 10A puts the three-dimensional object data on the auxiliaryline LN_01A in parallel with the ground on which the user moves his orher body.

In the example (B) in FIG. 33, the terminal apparatus 10A putsthree-dimensional object data on an auxiliary line LN_02A based on thejoint position JNT_01A in such a manner that the auxiliary line LN_02Ais overlaid on the joint position JNT_01A and is perpendicular to theground on which the user moves his or her body.

FIG. 34 shows an example screen presented to the user.

As shown in FIG. 34, in one screen 3403, the terminal apparatus 10Aarranges results of drawing of the avatar corresponding to the user froma plurality of points of view as the result of sensing of the movementof the body of the user, as shown in a left window 3403A, a front window3403B, and a right window 3403C.

For example, the terminal apparatus 10A arranges three-dimensionalobject data (what can be recognized as an avatar by the user)corresponding to the result of sensing of the movement of the body ofthe user (such as point cloud data) in the virtual space. In the virtualspace, the terminal apparatus 10A sets a plurality of virtual cameras(such as a virtual camera for the right window, a virtual camera for thefront window, and a virtual camera for the left window) with respect tothe three-dimensional object data corresponding to the result of sensingof the user. In this way, the terminal apparatus 10A can present theresults of drawing of the avatar corresponding to the user from aplurality of points of view to the user.

As shown in the drawing, the terminal apparatus 10A may not present theresult of estimation of the skeleton (a set of bones) to the user, whilepresenting the avatar. For example, the user probably has feweropportunities to see the result of estimation of the skeleton (a set ofbones) with his or her own eyes in daily life than those to see himselfor herself in the mirror or the like. Therefore, if the terminalapparatus 10A presents an avatar to the user but does not present theresult of estimation of the skeleton (a set of bones), the user will seethe avatar, which has a figure similar to that of himself or herselfseen with his or her own eyes in daily life, and can feel immersed inthe virtual space as if the avatar displayed on the screen was himselfor herself.

In the virtual space, the terminal apparatus 10A arrangesthree-dimensional object data on auxiliary lines LN_01A and LN_02A inassociation with the three-dimensional object data corresponding to theuser. Therefore, the terminal apparatus 10A can display the auxiliarylines along with the avatar corresponding to the user by performingrendering according to the settings of the virtual cameras.

Specifically, in the left window 3403A, the terminal apparatus 10A drawsthe auxiliary line LN_01A and the auxiliary line LN_02A in associationwith the avatar AVT_A and displays the auxiliary lines and the avatar tothe user. In the front window 3403B, the terminal apparatus 10A draws anauxiliary line LN_01B and an auxiliary line LN_02B in association withan avatar AVT_B and displays the auxiliary lines and the avatar to theuser. In the right window 3403C, the terminal apparatus 10A draws anauxiliary line LN_01C and an auxiliary line LN_02C in association withan avatar AVT_C and displays the auxiliary lines and the avatar to theuser.

(2-3) Methods of Evaluating the Movement of the Body of the User.

(i) As described above, the terminal apparatus 10A does not display theresult of estimation of the skeleton (a set of bones) to the user,although displaying the avatar. In this way, the terminal apparatus 10Amakes the user feel as if the avatar was a projection of himself orherself, thereby making the user concentrate on moving his or her body,training or the like. However, in evaluating the movement of the body ofthe user, the terminal apparatus 10A can use the result of estimation ofthe skeleton (a set of bones) of the user in order to evaluate whetherthe posture of the user is desirable or not or whether the movement ofthe body of the user is ideal or not for a type of movement of the body.

For example, in the example in FIG. 34, if the user desirably bends hisor her keens so that the thighs are vertical, the terminal apparatus 10Acan determine whether or not the posture of the user is desirable basedon the result of estimation of the knees of the skeleton (a set ofbones) of the user (that is, based on whether the thighs are vertical ornot).

(ii) The terminal apparatus 10A may evaluate the movement of the body ofthe user based on the three-dimensional object data (such as point clouddata) forming the avatar, which is the result of sensing of the movementof the body of the user, and the three-dimensional object datacorresponding to the auxiliary lines.

For example, the terminal apparatus 10A may evaluate whether themovement of the body of the user is ideal or not by determining whetherthe three-dimensional object data forming the avatar of the user (suchas the avatar AVT_A in the left window 3403A) and the three-dimensionalobject data corresponding to the auxiliary line (such as the auxiliaryline LN_01A in the left window 3403A) have collided with each other orcome close within a certain distance, for example.

When sensing the movement of the body of the user without using pointcloud data (such as when taking a two-dimensional image of the user),the terminal apparatus 10A may generate three-dimensional object datacorresponding to the body of the user based on the result of sensing(two-dimensional taken image data). The terminal apparatus 10A mayevaluate whether the movement of the body of the user is ideal or notbased on whether the three-dimensional object data and thethree-dimensional object data corresponding to the auxiliary line havecollided with each other or come close within a certain distance, forexample.

When sensing the movement of the body of the user without using pointcloud data (when taking a two-dimensional image of the user), theterminal apparatus 10A may estimate the posture (the positions of jointsor the skeleton) of the user in the two-dimensional image as describedabove (using a learned model that estimates the posture as describedabove), and draw an auxiliary line as object data in the two-dimensionalimage based on the estimation result. For example, from the positions ofjoints (such as the hip joints or the shoulder joints) of the user to beestimated in the two-dimensional image, the terminal apparatus 10Adetermines the position of the auxiliary line to be drawn as object datain the two-dimensional image according to the type of the movement ofthe user. The terminal apparatus 10A may evaluate whether the movementof the body of the user is ideal or not based on whether thetwo-dimensional image corresponding to the user and the auxiliary linedrawn as object data have collided with each other or come close withina certain distance, for example.

(2-4) Example Screens Presented to the User for Measuring SpecificMovements of the Body

(A) Standing Alignment

When the user is in a particular posture, such as a standing posture,the user can keep a good posture by making an adjustment not to lean tothe front, rear, left, or right.

FIG. 35 is a diagram showing an example screen for the user in astanding posture to adjust a forward, backward, left or right leaning ofthe body, displayed on the terminal apparatus 10A.

FIG. 36 is a diagram showing a process of determining the position of afigure object to be drawn that serves as a guide for movement of thebody of the user based on the result of estimation of the position of aparticular part of the user.

As shown in FIG. 35, in one screen 3500, the terminal apparatus 10Aarranges results of drawing of the avatar corresponding to the user froma plurality of points of view as the result of sensing of the movementof the body of the user, as shown in a left window 3501A and a frontwindow 3501B. In a window 3501C, the terminal apparatus 10A displays thepositions of a plurality of parts of the body of the user in such amanner that the positional relationship therebetween in a horizontalplane can be compared with each other.

In the left window 3501A, the terminal apparatus 10A displays an avatarAVT_35A corresponding to the user (which corresponds to the user viewedfrom the left side). The terminal apparatus 10A displays an auxiliaryline VL_35A as a figure object along with the avatar AVT_35A.

Here, a process in which the terminal apparatus 10A draws the auxiliaryline VL_35A will be described.

As shown in the example (A) in FIG. 36, the terminal apparatus 10Aestimates a position JNT_35D of the ankle (heel) as the particular partof the body of the user.

The terminal apparatus 10A can estimate the positions of the rightankle, the right foot, the right toe, the left ankle, the left foot, andthe left toe based on point cloud data obtained by measuring the body ofthe user (see the example in FIG. 12, for example). For example, theterminal apparatus 10A may determine the position of the auxiliary lineVL_35A as a figure object to be drawn so that the auxiliary line VL_35Ais perpendicular to the ground and passes through the result ofestimation of the position of the left or right ankle based on theresult of estimation of the position of the ankle.

In this way, the terminal apparatus 10A can allow the user to recognizethe positional relationship to the auxiliary line VL_35A while updatingthe display of the avatar AVT_35A each time the terminal apparatus 10Aobtains the result of sensing of the movement of the body of the user(that is, while drawing the avatar AVT_35A based on the point cloud dataobtained by measuring the body of the user in real time). The user canrecognize whether his or her body is misaligned to the front, rear,left, or right by comparing the avatar AVT_35A and the auxiliary lineVL_35A. For example, if the avatar AVT_35A significantly deviates fromthe auxiliary line VL_35A, the body of the user may be significantlymisaligned to the front, rear, left, or right.

When drawing the auxiliary line VL_35A in such a manner that the usercan compare the auxiliary line VL_35A with the avatar AVT_35A viewedfrom the left side as described above, the terminal apparatus 10A maydraw the auxiliary line based on the result of estimation of theposition of a part other than the ankle (heel) as the particular part ofthe body of the user (such as based on the result of estimation of theposition of the pelvis).

As shown in the front window 3501B in FIG. 35, the terminal apparatus10A displays an avatar AVT_35B corresponding to the user (whichcorresponds to the user viewed from the front). The terminal apparatus10A displays an auxiliary line LN_35A, an auxiliary line LN_35B, anauxiliary line LN_35C, and an auxiliary line VL_35B as figure objects inassociation with the avatar AVT_35B.

Here, a process in which the terminal apparatus 10A draws the auxiliaryline LN_35A and the like in the front window 3501B will be described.

As shown in the example (B) in FIG. 36, the terminal apparatus 10Aestimates a position JNT_35A of the head (the “head” in the example inFIG. 12), a position JNT_35B of the shoulder (the “center between theshoulders” in the example in FIG. 12), and a position JNT_35C of thehips (the “center of the pelvis”, the “right half of the pelvis”, or the“left half of the pelvis” in the example in FIG. 12) as the particularparts of the body of the user. The terminal apparatus 10A determines theposition of the auxiliary line VL_35B to be drawn so that the auxiliaryline VL_35B is perpendicular to the ground and passes through any of theposition JNT_35A of the head, the position JNT_35B of the shoulder, andthe position JNT_35C of the hip. The terminal apparatus 10A maydetermine the position of the auxiliary line VL_35B to be drawn based onthe position of a particular part other than these parts (such as a lineperpendicular to the ground that passes through the midpoint between theposition of the right heel and the position of the left heel).

Based on the result of estimation of the positions of these particularparts, the terminal apparatus 10A determines the positions of theauxiliary lines LN_35A and the like to be drawn. For example, as theposition of the auxiliary line LN_35A to be drawn, the terminalapparatus 10A determines a position at a predetermined distance (adistance DIST_35A in the example in FIG. 36) from the position JNT_35Aof the head, which is the result of estimation of the position of thehead of the user, in the horizontal direction (provided that thedirection perpendicular to the ground is the vertical direction). Theterminal apparatus 10A determines to draw the auxiliary line LN_35B at aposition at a predetermined distance from the position JNT_35B of theshoulder in the horizontal direction. The terminal apparatus 10Adetermines to draw the auxiliary line LN_35C at a position at apredetermined distance from the position JNT_35C of the hips in thehorizontal direction.

The terminal apparatus 10A draws these auxiliary lines LN_35A, LN_35B,and LN_35C in different ways (such as in different colors or shapes).Since the auxiliary lines are drawn in different ways, the user caneasily recognize to which part of the body of the user each auxiliaryline corresponds. In the window 3501C, the terminal apparatus 10A drawsauxiliary circles RD_35A, RD_35B, and RD_35C in the same ways as theauxiliary lines LN_35A, LN_35B, and LN_35C, respectively.

In the window 3501C, the terminal apparatus 10A draws an auxiliarycircle RD_35D, which is a figure object with respect to the positions ofthe heels of the user. The example (C) in FIG. 36 shows a case where theauxiliary circles RD_35A, RD_35B, and RD_35C are not drawn in the window3501C in FIG. 35. In the window 3501C, the terminal apparatus 10A drawsthe auxiliary circle RD_35A corresponding to the position of the head ofthe user, the auxiliary circle RD_35B corresponding to the shoulders,and the auxiliary circle RD_35C corresponding to the hip. The shape ofthese auxiliary circles RD_35A, RD_35B, and RD_35C may be the same as orsimilar to the shape of the auxiliary circle RD_35D. The terminalapparatus 10A displays these auxiliary circles RD_35A, RD_35B, andRD_35C at positions where the head, the shoulders, and the hips arelocated with respect to the particular reference part in the horizontalplane. In this way, the user can recognize in which direction, such asfront, back, left or right, his or her body leans and is prompted toalign the plurality of figure objects (the auxiliary circles RD_35A,RD_35B, and RD_35C) corresponding to a plurality of parts of the bodywith the reference auxiliary circle RD_35D.

As described above, the terminal apparatus 10A sequentially senses themovement of the body of the user as point cloud data or the like, andestimates the positions of at least three particular parts (four parts,the head, the shoulders, the hips, and the heels, in the example shownin FIG. 35 and FIG. 36 described above) of the user based on the resultof sensing of the movement of the body of the user. The terminalapparatus 10A designates any one of the at least three particular partsas a reference (the positions of the heels are designated as a referencein the example in FIGS. 35 and 36, although any other part along thebody axis, such as the hips, can be designated as a reference),determines the positions where the figure objects indicating thepositional relationship between the other parts in the horizontal planeare to be drawn, and displays the figure objects at the determinedpositions.

In an object 3501D in FIG. 35, the terminal apparatus 10A displays anindex of the time left before the expiration of the period for the userto adjust the front-rear or left-right balance of his or her body in thestanding posture. In this way, the user is prompted to adjust thefront-rear or left-right balance within the remaining time as far aspossible, and the posture of the user can be further improved.

(B) Lateral Arm Elevation

The appropriateness of the posture of the user can be evaluated bymeasuring a movement of the user in the standing posture raising botharms straight out to his or her sides. The movement of lateral armelevation will be described, provided that the timing when the user inthe standing posture has completely raised his or her arms is the timingof a peak of the movement.

FIG. 37 is a diagram showing an example screen for the terminalapparatus 10A to prompt the user to make the movement of lateral armelevation.

FIG. 38 is a diagram showing a process of determining the position of afigure object to be drawn that serves as a guide for movement of thebody of the user based on the result of estimation of the position of aparticular part of the user.

FIG. 39 is a diagram showing a course of movement of the body of theuser.

FIG. 40 is a diagram showing a state of the user having completelyraised both arms at the timing of the peak of the movement of lateralarm elevation.

FIG. 41 is a diagram showing an example in which a result of sensing ofthe user at the timing of the peak of the movement of lateral armelevation is saved and displayed as an avatar for comparison.

As shown in FIG. 37, before starting the movement of lateral armelevation, the user assumes an initial posture, which is a standingposture with both arms stuck to the sides of his or her body. Theterminal apparatus 10A may notify the user to assume the initial posturethrough audio or image, for example.

The terminal apparatus 10A draws a figure object based on the initialposture of the user and the result of estimation of the position of aparticular part. In the example shown, the terminal apparatus 10Adisplays an avatar AVT_37A corresponding to the user (which correspondsto the user viewed from the left side) in a left window 3701A of ascreen 3700. The terminal apparatus 10A draws the following figureobjects in the left window 3701A.

-   -   An auxiliary line VL_37A that is a figure object that makes a        front-rear misalignment of the user perceptible. This auxiliary        line is fixed and does not move while the user is moving his or        her body.    -   An auxiliary line VL_37D that indicates the position of the        front surface of the head of the user in the initial posture.        While the user is making the movement of lateral arm elevation,        the user can more easily recognize whether the body of the user        is moving forward or rearward by watching the head and        perceiving the positional relationship between the avatar        AVT_37A, which is the result of sensing of the movement of the        body of the user, and the auxiliary line VL_37D. This auxiliary        line is fixed and does not move while the user is moving his or        her body.    -   An auxiliary line VL_37E that indicates the position of the        surface of the abdomen of the user in the initial posture. While        the user is making the movement of lateral arm elevation, the        user can more easily recognize whether the body of the user is        moving forward or rearward by watching the abdomen and        perceiving the positional relationship between the avatar        AVT_37A, which is the result of sensing of the movement of the        body of the user, and the auxiliary line VL_37E. This auxiliary        line is fixed and does not move while the user is moving his or        her body.    -   An auxiliary line HL_37A that indicates the distance between the        auxiliary line VL_37A and the auxiliary line VL_37D. This        auxiliary line is fixed and does not move while the user is        moving his or her body.    -   An auxiliary line HL_37B that makes the positions of the        shoulders of the user in the initial posture more recognizable.        This auxiliary line is fixed and does not move while the user is        moving his or her body.    -   An auxiliary line HL_37C that indicates the distance between the        auxiliary line VL_37A and the auxiliary line VL_37E. This        auxiliary line is fixed and does not move while the user is        moving his or her body.

In a right window 3701C, the terminal apparatus 10A displays auxiliarylines VL_37C, VL_37F, VL_37G, HL_37D, HL_37E, and HL_37F along with anavatar AVT_37C (which corresponds to the user viewed from the rightside).

The auxiliary line VL_37C corresponds to the auxiliary line VL_37A.

The auxiliary line VL_37F corresponds to the auxiliary line VL_37D.

The auxiliary line VL_37G corresponds to the auxiliary line VL_37E.

The auxiliary line HL_37D corresponds to the auxiliary line HL_37A.

The auxiliary line HL_37E corresponds to the auxiliary line HL_37B.

The auxiliary line HL_37F corresponds to the auxiliary line HL_37C.

In a front window 3701B, the terminal apparatus 10A displays an avatarAVT_37B (which corresponds to the user viewed from the front). In thefront window 3701B, the terminal apparatus 10A displays the followingfigure objects.

-   -   An auxiliary line VL_37B that is a figure object that makes a        left-right misalignment of the user perceptible. This auxiliary        line is fixed and does not move while the user is moving his or        her body.    -   An auxiliary line LN_37L (which corresponds to the left arm of        the user) and an auxiliary line LN_37R (which corresponds to the        right arm of the user) that are associated with the avatar        AVT_37B to make whether the elbow is bent while the user is        making the movement of lateral arm elevation perceptible. The        terminal apparatus 10A updates the position of each drawn figure        object in association with the result of sensing of the movement        of the body of the user. That is, the terminal apparatus 10A        updates the positions of the drawn auxiliary lines LN_37L and        LN_37R in association with the movement of the avatar.    -   A polygon QB_37A (which corresponds to the left arm) that        indicates the end point of the auxiliary line LN_37L and a        polygon QB_37B (which corresponds to the right arm) that        indicates the end point of the auxiliary line LN_37R. If the        user bends his or her elbow during the movement of lateral arm        elevation, the polygon QB_37B or QB_37A is drawn at a position        outside the avatar AVT_37B. Therefore, the user can easily        recognize a bend of his or her elbow by recognizing the        positional relationship between the avatar AVT_37B and the        polygon QB_37A or QB_37B.    -   A polygon QB_37C (which corresponds to the left arm) and a        polygon QB_37D (which corresponds to the right arm) that allow        the user to perceive whether the user is shrugging his or her        shoulders (that is, whether the user is properly using the        shoulder joints to raise his or her arms) when the user has        completely raised his or her arms. As described later, if the        user is not shrugging his or her shoulders when the user has        completely raised his or her arms, the terminal apparatus 10A        draws the polygon QB_37C for the left arm at a position where        the polygon QB_37C overlaps with the polygon QB_37A and draws        the polygon QB_37D for the right arm at a position where the        polygon QB_37D overlaps with the polygon QB_37B.

When prompting the user to make the movement of lateral arm elevation,the terminal apparatus 10A displays an icon 3701D that indicates thatthe user is in the initial posture, an icon 3701E that indicates theuser is making the movement of lateral arm elevation, and an icon 3701Fthat indicates that the measurement has been finished.

A process of drawing these figure objects will be described.

As shown in the example (A) in FIG. 38, as particular parts of the bodyof the user, the terminal apparatus 10A estimates a position JNT_38B ofthe hips (such as the “center of the pelvis” in the example in FIG. 12),a position JNT_38C between the hips and the shoulders (such as the“thoracic and lumbar spines” in the example in FIG. 12), a positionJNT_37D of the shoulders, and a position JNT_38E of the head.

The terminal apparatus 10A determines the position of the auxiliary lineVL_37A to be drawn so that the auxiliary line is perpendicular to theground and passes through the positions of the position JNT_38B of thehips and the other positions described above.

The terminal apparatus 10A determines the position of the auxiliary lineVL_37D to be drawn based on the position where a line passing throughthe position JNT_38E of the head and extending from the back to thefront of the user in the horizontal direction intersects with the frontsurface of the body of the user. That is, the terminal apparatus 10Adetermines the position of the figure object to be drawn based on theresult of estimation of the positions of parts of the body and theresult of sensing of the movement of the body of the user as point clouddata.

The terminal apparatus 10A determines the position of the auxiliary lineVL_37E to be drawn based on the position where a line passing throughthe central position between the position JNT_38B of the hips and theposition JNT_38C between the hips and the shoulders and extending fromthe back to the front of the user in the horizontal direction intersectswith the front surface of the body of the user.

The method of drawing each auxiliary line in the example (C) can be thesame as that in the example (A).

As shown in the example (B), the terminal apparatus 10A draws theauxiliary line VL_37B based on the result of estimation of the positionsof parts of the body of the user, such as the hips and the pelvis. Theterminal apparatus 10A may draw the auxiliary line VL_37B based on theresult of estimation of the position of the heel of the right foot andthe position of the heel of the left foot of the user in such a mannerthat the auxiliary line VL_37B passes through the central positionbetween these positions and is perpendicular to the ground.

The terminal apparatus 10A determines the position of the auxiliary lineLN_37L to be drawn based on the result of estimation of a positionJNT_38F of a wrist (left wrist) of the user and a position JNT_38G of anelbow (left elbow) of the user in such a manner that the auxiliary lineLN_37L starts at the left wrist and ends at a point (the polygon QB_37A)on the extension of the line connecting the left wrist and the elbow atthe same distance from the elbow as the length between the left wristand the elbow.

The terminal apparatus 10A determines the position of the auxiliary lineLN_37R to be drawn based on the result of estimation of a positionJNT_381 of a wrist (right wrist) of the user and a position JNT_38J ofan elbow (right elbow) of the user in such a manner that the auxiliaryline has a length determined in the same manner as the auxiliary lineLN_37L.

Based on a position JNT_38H of a shoulder (left shoulder), the terminalapparatus 10A determines coordinates at a predetermined distance fromthe body of the user in an outward and upward direction. The coordinatesare set to correspond to the position of the shoulder at the time whenthe peak of the movement of lateral arm elevation of the user isreached. The terminal apparatus 10A may determine the coordinates basedon the size of body of the user (such as the height or the shoulderwidth of the user). Based on the determined coordinates, the terminalapparatus 10A determines the position of the polygon QB_37C to be drawn.

In the same manner as that for determining the position of the polygonQB_37C to be drawn, the terminal apparatus 10A determines the positionof the polygon QB_37D to be drawn based on a position JNT_38K of ashoulder (right shoulder) of the user.

As shown in FIG. 39, when the user makes a movement of raising his orher arms, the terminal apparatus 10A displays trajectories of movementof the hands as a trajectory LN_39A (a trajectory of movement of theleft hand) and a trajectory LN_39C (a trajectory of movement of theright hand) based on the history of the result of sensing of themovement of the body of the user and the history of the determinedpositions of the wrists.

Suppose that the user has completely raised his or her arms and reachedthe peak of the movement as shown in FIG. 40. The terminal apparatus 10Adetects that the peak of the movement of lateral arm elevation of theuser is reached (for example, detects that the user has completelyraised his or her arms based on the result of estimation of the positionof a particular part of the user), and draws an avatar corresponding tothe result of sensing (the posture of the user) at this timing. Here, ifthe polygon QB_37A drawn and updated in association with the movement ofthe body of the user and the polygon QB_37C serving as an index of theposture with both arms completely raised are in a particular positionalrelationship (overlap with each other, for example), at least any of thepolygon QB_37A and the polygon QB_37C may be displayed in a differentmanner than when the polygons do not overlap with each other. In thisway, the terminal apparatus 10A can make the user easily perceive thatthe movement of lateral arm elevation is proper (that the user is notshrugging his or her shoulders, for example). Similarly, when thepolygon QB_37B and the polygon QB_37D are in a particular positionalrelationship, the terminal apparatus 10A can display the polygons in adifferent manner than when the polygons are not in the particularpositional relationship.

As shown in FIG. 41, the terminal apparatus 10A displays a historyavatar AVT_37D and a history avatar AVT_37E side by side with the avatarAVT_37A and the avatar AVT_37C drawn and updated in association with theresult of sensing of the movement of the body of the user, respectively.

As shown in FIG. 41, since the avatars with the arms completely raisedin the movement of lateral arm elevation of the user, the trajectories(the trajectories LN_39A and LN_39C) of the movement of the user raisingthe arms, and the avatars AVT_37A and AVT_37C representing the postureof the user in real time are displayed side by side, the user can beprovided with a material based on which the user can think how to make amovement.

(C) Squat

FIG. 42 is a diagram showing an example screen for the terminalapparatus 10A to prompt the user to make a squat movement.

FIG. 43 is a diagram showing a process of determining the position of afigure object to be drawn that serves as a guide for movement of thebody of the user based on the result of estimation of the position of aparticular part of the user.

FIG. 44 is a diagram showing a course of movement of the body of theuser.

FIG. 45 is a diagram showing a state of the user having completelysquatted down at the timing of the peak of the squat movement of theuser.

FIG. 46 is a diagram showing an example in which a result of sensing ofthe user at the timing of the peak of the squat movement is saved anddisplayed as an avatar for comparison.

As shown in FIG. 42, in an initial posture before starting the squatmovement, the terminal apparatus 10A draws a figure object based on theinitial posture and the result of estimation of the position of aparticular part.

In the example shown, in a left window 4201A of a screen 4200, theterminal apparatus 10A draws an avatar AVT_42A and at the same timedraws the following auxiliary lines.

-   -   An auxiliary line LN_42A for visualizing the angle of the back        of the user.    -   An auxiliary line LN_42B for visualizing the angle of the shank        of the user.

In a right window 4201C, the terminal apparatus 10A draws an avatarAVT_42C and at the same time draws the following auxiliary line.

-   -   An auxiliary line VL_42C for allowing the user to perceive        whether the posture of the user is leaning forward or backward.        This auxiliary line is displayed at a fixed position while the        movement of the body of the user is measured.

In a front window 4201B, the terminal apparatus 10A draws an avatarAVT_42B and at the same time draws the following auxiliary lines.

-   -   An auxiliary line VL_42B for allowing the user to perceive        whether the posture of the user is leaning to the left or right.        This auxiliary line is displayed at a fixed position while the        movement of the body of the user is measured.    -   An auxiliary line LN_37L for allowing the user to perceive        whether the left foot of the user is misaligned to the left or        right.    -   An auxiliary line LN_37R for allowing the user to perceive        whether the right foot of the user is misaligned to the left or        right.

The terminal apparatus 10A displays an object 4201D indicating a statebefore starting the squat movement, an object 4201E indicating that thesquat movement is being measured, and an object 4201F indicating thatthe measurement is finished.

As shown in the example (A) in FIG. 43, as the result of estimation ofthe positions of parts of the user, the terminal apparatus 10A draws, asthe auxiliary line LN_42B, a line segment starting at a position JNT_43Aof the heel and passing through a position JNT_43B of the knee based onthe position JNT_43A of the heel and the position JNT_43B of the knee.

The terminal apparatus 10A draws, as the auxiliary line LN_42A, a linesegment starting at a position JNT_43C of the hips and passing through aposition JNT_43D of the shoulder based on the position JNT_43C of thehips (such as the pelvis) and the position JNT_43D of the shoulder.

The terminal apparatus 10A may determine the lengths of these auxiliarylines in such a manner that the end points of the auxiliary lines LN_42Aand LN_42B are at the same height.

As shown in the example (C), the terminal apparatus 10A determines theposition of the auxiliary line VL_42C to be drawn so that the auxiliaryline is perpendicular to the ground based on a position JNT_43J of theheel. The terminal apparatus 10A may draw the auxiliary line VL_42Cbased on the position of the hips or the like, rather than the positionJNT_43J of the heel.

As shown in the example (B), based on a position JNT_43E of a heel (leftheel) and a position JNT_43F of a knee (left knee), the terminalapparatus 10A determines the position of the auxiliary line LN_37L to bedrawn, which starts at the position of the heel and ends at a point onthe extension of the line connecting the heel and the knee at the samedistance from the knee as the measured length between the heel and theknee.

The terminal apparatus 10A determines the position of the auxiliary lineVL_42B to be drawn so that the auxiliary line is perpendicular to theground and passes through a position JNT_431 of the hip. The terminalapparatus 10A may determine the position of the auxiliary line VL_42B tobe drawn based on the positions of the left and right heels (JNT_43E andJNT_43G) so that the auxiliary line is perpendicular to the ground andpasses through the midpoint between these positions.

As shown in FIG. 44, when the user starts the squat movement, theterminal apparatus 10A draws a trajectory LN_42M of the squat of theuser in the front window 4201B. For example, the terminal apparatus 10Amay draw the trajectory LN_42M by retaining the result of estimation ofthe position of the neck (or the head or the shoulder) of the user as ahistory.

In the right window 4201C, the terminal apparatus may display atrajectory LN_42N of the squat of the user. For example, the terminalapparatus 10A may draw the trajectory LN_42N by retaining the result ofestimation of the position of the neck or the like of the user as ahistory.

In this way, the user can easily recognize whether the user is squattingwithout fluctuating to the left or right by comparing the trajectoryLN_42M with the auxiliary line VL_42B, and whether the user is squattingwithout fluctuating to the front or back by comparing the trajectoryLN_42N with the auxiliary line VL_42C.

As shown in FIG. 45, when the user has completely squatted down, thepeak of the squat movement is reached. In response to detecting that thepeak of the squat movement of the user is reached (for example, inresponse to detecting that the peak is reached by comparing the resultof estimation of the position of a particular part with the position ofthe part in the posture at the time when the peak of the squat movementof the user is reached), the terminal apparatus 10A saves the sensingresult (such as point cloud data) at the timing of the peak.

In the state where the user has completely squatted down, the terminalapparatus 10A may determine whether the auxiliary line LN_42B (whichindicates the angle of the knee) and the auxiliary line LN_42A (whichindicates the angle of the back) are diverging. In the example in FIG.45, the auxiliary line LN_42B and the auxiliary line LN_42A arediverging with respect to the respective starting points (the heel andthe hip), so that it can be determined that the user is not leaningforward and in a good posture. The terminal apparatus 10A may draw theauxiliary lines LN_42B and LN_42A in such a manner that the end pointsof the auxiliary line LN_42B (starting at the heel) and the auxiliaryline LN_42A (starting at the hip) are at the same height from the groundin the state where the user has completely squatted down. The terminalapparatus 10A may determine whether the movement of the user is goodbased on whether the distance between the end point of the auxiliaryline LN_42A and the end point of the auxiliary line LN_42B is equal toor greater than a certain distance (the movement of the user isdetermined to be good when the distance between the end points is equalto or greater than the certain distance).

As shown in FIG. 46, when the user stands up after completely squattingdown, the terminal apparatus 10A displays the result of sensing of theuser in the squat position as an avatar AVT_42D in the left window4201A, as an avatar AVT_42E in the front window 4201B, and as an avatarAVT_42F in the right window 4201C.

In the left window 4201A, the terminal apparatus 10A displays theauxiliary lines LN_42A and LN_42B in the squat position along with theavatar AVT_42D.

In the front window 4201B, the terminal apparatus 10A draws a trajectoryLN_42P of the standing up of the user after completely squatting down(that is, after the timing of the peak is reached) (for example, thetrajectory is drawn based on the position of the neck).

In the right window 4201C, the terminal apparatus 10A draws a trajectoryLN_42Q of the standing up of the user after completely squatting down(that is, after the timing of the peak is reached) (for example, thetrajectory is drawn based on the position of the neck).

(D) A movement of lowering the hips while stepping a foot forward.

FIG. 47 is a diagram showing an example screen for the terminalapparatus 10A to prompt the user to make a movement of lowering his orher hips while stepping a foot forward.

FIG. 48 is a diagram showing an example screen in a situation where itis indicated to the user that the initial posture of the user is proper.

FIG. 49 is a diagram showing a situation where the measurement of themovement of the user is started.

FIG. 50 is a diagram showing a situation where whether the movement ofthe user is proper or not is determined, and the determination result ispresented to the user.

FIG. 51 is a diagram showing a situation where whether the movement ofthe user is proper or not is determined, and the determination result ispresented to the user.

As shown in FIG. 47, in a screen 4700, the terminal apparatus 10Adisplays an avatar AVT_47B corresponding to the user (which correspondsto the user viewed from the front) in a front window 4701B. In a rightwindow 4701C, the terminal apparatus 10A displays an avatar AVT_47Ccorresponding to the user (which corresponds to the user viewed from theright) and an auxiliary line LN_47A and an auxiliary line LN_47B servingas a guide for the foot positions of the user.

In the example shown, the terminal apparatus 10A displays anarrow-shaped object that prompts the user to step the back footbackward.

In the example in FIG. 47, the terminal apparatus 10A measures thelength of the leg of the user based on the result of estimation of theposition of a particular part of the user. For example, the terminalapparatus 10A measures the length of the leg of the user based on theposition of the heel and the position of the hips (pelvis) of the user.In this way, the terminal apparatus 10A determines a required steplength for the movement of the user. The terminal apparatus 10A mayadjust the required step length in accordance with the physique (such asthe length of the leg) of the user. The terminal apparatus 10Adetermines the distance between the auxiliary line LN_47A and theauxiliary line LN_47B based on the result of measurement of the lengthof the leg of the user, and displays the auxiliary lines in the rightwindow 4701C.

In the example shown, in the screen 4700, the terminal apparatus 10Adisplays an object 4701D for visualizing the rotation of the hips andthe rotation of the shoulders of the user. The terminal apparatus 10Adetermines the rotation of the hips of the user based on the result ofestimation of the position of the pelvis (the positions of the “lefthalf of the pelvis” and the “right half of the pelvis” in the example inFIG. 12). The terminal apparatus 10A determines the rotation of theshoulders of the user based on the result of estimation of the positionsof the shoulders (the “left shoulder” and the “right shoulder” in theexample in FIG. 12).

In the object 4701D, the terminal apparatus 10A displays an auxiliaryline LN_47SB indicating the rotation of the hips and an auxiliary lineLN_47SH indicating the rotation of the shoulders with respect to theposition of the hips (such as the center of the pelvis), for example. Inthis way, the user can easily perceive how much the hips and theshoulders are rotating in the movement of lowering the hips whilestepping a foot forward.

As shown in FIG. 48, before measuring the movement of the user taking abig step forward, lowering his or her hips, and then moving his or herhips up and down, the terminal apparatus 10A determines whether the userhas taken an adequate step or not.

The terminal apparatus 10A may determine that the user has taken anappropriate step or not in the following cases.

-   -   A case where the angle formed by the legs (such as the angle        formed by a line segment extending from the left knee to the        pelvis and a line segment extending from the right knee to the        pelvis) based on the result of estimation of the positions of        parts of the body of the user is equal to or greater than a        predetermined angle.    -   A case where a particular part (such as the heel) of a foot of        the user having taken a step and the auxiliary line LN_47A or        LN_47B are in a predetermined positional relationship. For        example, a case where the result of estimation of the position        of the heel of the front foot of the user and the auxiliary line        LN_47A overlap with each other or are within a certain distance.        As another example, a case where the result of estimation of the        position of the heel of the back foot of the user and the        auxiliary line LN_47B overlap with each other or are within a        certain distance.

When the front foot and the back foot are at proper positions, theterminal apparatus 10A displays an object OBJ_47A and an object OBJ_47Bindicating as such, respectively. If a foot is not at the properposition, the terminal apparatus 10A displays as such (such as “NG”) inthe object OBJ_47A or the object OBJ_47B.

The terminal apparatus 10A may draw the object OBJ_47A or the objectOBJ_47B based on the result of estimation of the position of aparticular part of the user. For example, the terminal apparatus 10A maydraw the object OBJ_47A in association with coordinates that are theresult of estimation of the position of the heel of the user (forexample, the object OBJ_47A in the form of a speech bubble may be drawnwith respect to the position of the heel of the user).

As described above, in response to detecting that the posture of theuser before starting the movement is proper, the terminal apparatus 10Amay makes a notification to prompt the user to start the movement (forexample, a notification that suggests specific details of the movementto the user, such as “Slowly lower your hips and return to the originalposture”) through audio, an image or the like. In this way, the user canconfirm that his or her posture before starting the movement is proper,and can easily start the movement of moving his or her hips up and down.

As shown in FIG. 49, in a phase where the user makes a movement ofmoving his or her hips up and down, the terminal apparatus 10A drawsauxiliary lines VL_47B and VL_47E in a fixed manner (at fixed positions)regardless of the movement of the user along with the avatar AVT_47Bcorresponding to the user in the front window 4701B.

The auxiliary line VL_47B is an auxiliary line that allows the user toperceive whether the user is fluctuating to the left or right while theuser is making the movement of moving his or her hips up and down. Forexample, the terminal apparatus 10A draws the auxiliary line VL_47Bbased on the result of estimation of the position of the hips of theuser before the user starts the movement of lowering his or her hips.

The auxiliary line VL_47E is an auxiliary line that allows the user toperceive whether a foot (such as the front foot) of the user isfluctuating to the left or right while the user is moving his or herhips up and down. For example, the terminal apparatus 10A draws theauxiliary line VL_47B based on the result of estimation of the positionsof the heel and knee of the front leg of the user before the user startsthe movement of lowering his or her hips.

An auxiliary line VL_47F is an auxiliary line that allows the user toperceive whether the shoulders of the user are fluctuating to the leftor right while the user is moving his or her hips up and down. Theterminal apparatus 10A draws the auxiliary line VL_47F in associationwith the movement of the user lowering his or her hips.

In the right window 4701C, the terminal apparatus 10A displays auxiliarylines VL_47C and LN_47C in a fixed manner regardless of the movement ofthe user along with the avatar AVT_47C corresponding to the user.

The auxiliary line VL_47C is an auxiliary line that allows the user toperceive whether the user is vertically lowering his or her hips withoutfluctuating to the front or back while the user is moving his or herhips up and down. For example, the terminal apparatus 10A draws theauxiliary line VL_47C based on the result of estimation of the positionof the pelvis or the like of the user before the user starts themovement of lowering his or her hips.

The auxiliary line LN_47C is an auxiliary line that allows the user toperceive whether the user is lowering his or her hips too deeply in theforward or backward direction (such as whether the front knee is movingtoo forward) while the user is moving his or her hips up and down. Forexample, the terminal apparatus 10A draws the auxiliary line LN_47Cbased on the result of estimation of the position of the heel or thelike of the user before the user starts the movement of lowering his orher hips.

The terminal apparatus 10A draws an object OBJ_47E and an object OBJ_47Fserving as a guide for the position of a particular part of the body ofthe user while the user is lowering his or her hips and the speed oflowering of his or her hips in synchronization with the movement of theuser moving his or her hips up and down. These objects OBJ_47E andOBJ_47F move along the auxiliary line VL_47C in synchronization with themovement of the user moving his or her hips up and down.

The object OBJ_47E is an object that allows the user to perceive areference position for the neck or shoulders while the user is movinghis or her hips up and down.

The object OBJ_47F is an object that allows the user to perceive areference position for the hips while the user is moving his or her hipsup and down.

Suppose that the user is slightly leaning forward when the user islowering his or her hips as shown in FIG. 50. As shown in the drawing,in the right window 4701C, the terminal apparatus 10A draws an objectOBJ_47J that indicates the actual position of a particular part of thebody of the user (the result of estimation of the position of the neck,the shoulder, the head or the like) for comparison with the objectOBJ_47E serving as a guide for an ideal movement.

The terminal apparatus 10A draws an object OBJ_47K that indicates theactual position of a particular part of the body of the user (the resultof estimation of the position of the hips, the pelvis or the like) forcomparison with the object OBJ_47F serving as a guide for an idealmovement.

In order to allow the user to easily perceive whether the user isvertically moving his or her hips up and down, when the object OBJ_47Eand the object OBJ_47J are in a predetermined positional relationship(for example, overlap with each other or are within a certain distance),the terminal apparatus 10A may draw the objects in a different mannerthan when the objects are not in the predetermined positionalrelationship. Similarly, when the object OBJ_47F and the object OBJ_47Kare in a predetermined positional relationship, the terminal apparatus10A may draw the objects in a different manner than when the objects arenot in the predetermined positional relationship. In this way, the usercan easily perceive in real time whether the user is moving his or herhips up and down in an ideal manner or whether the user is moving his orher body by following the guide for an ideal movement.

The terminal apparatus 10A determines whether the front leg is properlymoving or not, and displays the determination result through an objectOBJ_47P. For example, the terminal apparatus 10A may determine whetherthe front leg is properly moving or not as follows.

-   -   Whether the user is properly moving the front leg (such as        whether the front leg is not moving too forward) is determined        based on the angle of a particular part of the front leg, such        as the heel, the knee or the pelvis, of the user.    -   Whether the result of estimation of the position of a part (such        as the heel or the knee) of the lower part of the body of the        user and the auxiliary line LN_47C are in a predetermined        positional relationship is determined. (For example, if the        position of the heel or knee is located backward of the        auxiliary line LN_47C, the user is determined to be leaning        forward. For example, if a line segment connecting the position        of the heel and the position of the knee (that is, the angle of        the knee) forms an angle equal to or smaller than a        predetermined angle with the auxiliary line LN_47C, the user is        determined to be properly lowering his or her hips.)

The terminal apparatus 10A may determine the position of the objectOBJ_47P to be drawn based on the result of estimation of the position ofa part of the user. For example, the terminal apparatus 10A may draw theobject OBJ_47P with respect to the position of the knee of the user. Inthat case, the drawn object OBJ_47P moves as the user moves his or herbody in real time, so that the user can easily perceive whether the useris properly moving his or her body.

The terminal apparatus 10A determines whether the back leg is properlymoving or not, and displays the determination result through an objectOBJ_47N. The terminal apparatus 10A can determine whether the back footis properly moving or not in the same manner as with the front legdescribed above.

The terminal apparatus 10A determines whether the front leg isfluctuating to the left or right or not, and displays the determinationresult through an object OBJ_47M in the front window 4701B. For example,based on the result of estimation of the positions of the heel and theknee, the terminal apparatus 10A may determine that the front leg isfluctuating to the left or right when the line of the shank is notvertical (for example, when the line of the shank forms an angle equalto or greater than a predetermined angle with the auxiliary lineVL_47E). In the example in FIG. 50, the front leg is not fluctuating tothe left or right, and “OK” is displayed in the object OBJ_47M.

Referring to FIG. 51, in the right window 4701C, the user is properlymoving his or her hips up and down, and “OK” is displayed in the objectOBJ_47P and the object OBJ_47N.

(3) APPENDICES

The following appendices are a summary of what has been described in theabove embodiments.

Appendix 1

A program that makes a computer operate, wherein the program makes aprocessor of the computer perform: a step of sequentially sensing amovement of a body of a user; a step of presenting, to the user, anavatar corresponding to the user, rather than a photographed image of anappearance of the user, based on a result of the sensing so that theuser perceives a shape of a surface of the body of the user; a step ofdetermining a position of a figure object to be drawn that serves as aguide for movement of the body of the user in association with theavatar based on the result of the sensing; and a step of displaying thefigure object at the determined position.

Appendix 2

The program according to Appendix 1, wherein in the step of presentingthe avatar, the avatar is presented to the user with a reduced amount ofinformation compared with the photographed image.

Appendix 3

The program according to Appendix 1 or 2, wherein in the step ofpresenting the avatar, the avatar is presented to the user each time theresult of the sensing is obtained.

Appendix 4

The program according to any one of Appendices 1 to 3, wherein in thestep of presenting the avatar, an avatar corresponding to the userviewed in a first direction and an avatar corresponding to the userviewed in a second direction that is different from the first directionare drawn side by side and presented to the user.

Appendix 5

The program according to any one of Appendices 1 to 4, wherein in thestep of sensing, the movement of the body of the user is sensed as pointcloud data, and in the step of presenting the avatar, the avatar isdrawn as a set of graphic elements based on point cloud data andpresented to the user.

Appendix 6

The program according to any one of Appendices 1 to 5, wherein theprogram makes the processor further perform: a step of estimatingpositions of a plurality of particular parts of the user based on theresult of the sensing of the movement of the body of the user; and astep of drawing the avatar while not drawing a result of the estimationof the positions of the particular parts, and evaluating a way ofmovement of the body of the user based on the result of the estimationof the positions of the particular parts.

Appendix 7

The program according to any one of Appendices 1 to 6, wherein in thestep of presenting the avatar, the avatar drawn is updated in responseto the result of the sensing of the movement of the body of the user,and in the step of displaying, the figure object is displayed at a fixedposition regardless of the movement of the body of the user.

Appendix 8

The program according to Appendix 7, wherein the program makes theprocessor further perform: a step of estimating a position of aparticular part of the user based on the result of the sensing of themovement of the body of the user; and a step of drawing, along with theavatar, a time-series history of positions of a first part of the userobtained by the sensing, the first part being a subject of themeasurement of the movement of the body of the user.

Appendix 9

The program according to Appendix 7, wherein in the step of determining,it is determined to draw a linear figure object in parallel with a bodyaxis of the user at a fixed position regardless of the movement of thebody of the user based on the result of the sensing of the user.

Appendix 10

The program according to Appendix 1, wherein in the step of determining,the position of the figure object to be drawn that serves as a guide forthe user to make a particular movement is determined based on a resultof sensing of a first posture of the user before starting the particularmovement.

Appendix 11

The program according to Appendix 10, wherein the particular movement isa predetermined movement for measurement of the movement of the body ofthe user, and in the step of determining, the position of the figureobject to be drawn that serves as a guide for movement of the body ofthe user at a predetermined timing in the particular movement isdetermined based on the first posture.

Appendix 12

The program according to Appendix 10, wherein in the step ofdetermining, the position of the figure object to be drawn that servesas a guide for at least one of a group consisting of a front-rearbalance and a left-right balance of the body of the user is determinedbased on the first posture.

Appendix 13

The program according to Appendix 10, wherein in the step ofdetermining, the position of the figure object to be drawn that servesas a guide for a second posture that can be assumed by the user whilethe user is making the particular movement is determined based on thefirst posture.

Appendix 14

The program according to Appendix 13, wherein the particular movement isa movement made by the user in a state where the user has taken a bigstep forward, and in the step of determining, the position of the figureobject to be drawn that serves as a guide for a posture of the user inwhich the user has stepped a front foot forward is determined based onthe first posture.

Appendix 15

The program according to Appendix 14, wherein the program makes theprocessor further perform a step of estimating a position of a lowerlimb of the user based on the result of the sensing of the movement ofthe body of the user, and in the step of determining, the position ofthe figure object to be drawn that serves as a guide for a step positionof the user having taken a big step forward is determined based on theresult of the estimation.

Appendix 16

The program according to Appendix 13, wherein the program makes theprocessor further perform: a step of estimating a position of aparticular part of the user based on the result of the sensing of themovement of the body of the user; and a step of evaluating whether theuser is properly assuming the second posture based on a positionalrelationship between the estimated position of the particular part andthe position of the figure object.

Appendix 17

The program according to Appendix 16, wherein when it is evaluated thatthe user is properly assuming the second posture in the step ofevaluating, the user is presented as such.

Appendix 18

The program according to Appendix 16, wherein the particular movementinvolves a body movement made by the user in the second posture, and inresponse to evaluation that the user is properly assuming the secondposture in the step of evaluating, measurement of the particularmovement is started, and the start of the measurement is presented tothe user.

Appendix 19

The program according to Appendix 1, wherein the program makes theprocessor further perform: a step of estimating positions of a pluralityof particular parts of the user based on the result of the sensing ofthe movement of the body of the user, and in the step of determining, aposition of a linear figure object to be drawn that allows the user toperceive whether the particular parts are properly moving in themovement of the body of the user is determined based on the estimatedpositions of the particular parts.

Appendix 20

The program according to Appendix 19, wherein in the step of displaying,a line segment that is an extension of a line extending from aperipheral part of a particular part to the particular part is displayedas the figure object along with the avatar.

Appendix 21

The program according to Appendix 20, wherein in the step of displaying,a figure object having a large size compared with a width of the linesegment is drawn and displayed at an end point of the line segment.

Appendix 22

The program according to Appendix 21, wherein in the step of displaying,the figure object corresponding to the end point of the line segment isdisplayed in such a manner that the figure object moves in associationwith the result of the sensing of the movement of the body of the user,and a way of display of the figure object is changed based on apositional relationship between the figure object arranged in a fixedmanner regardless of the movement of the body of the user and the figureobject corresponding to the end point.

Appendix 23

The program according to Appendix 19, wherein the program makes theprocessor further perform: a step of estimating positions of a pluralityof particular parts of the user based on the result of the sensing ofthe movement of the user, and in the step of displaying, a figure objectindicating an inclination of a first part of the user and a figureobject indicating an inclination of a second part that is different fromthe first part are drawn and displayed in association with the firstpart and the second part, respectively.

Appendix 24

The program according to Appendix 19, wherein the program makes theprocessor further perform: a step of estimating a position of aparticular part of the user based on the result of the sensing of themovement of the body of the user, and in the step of determining, theposition of the figure object to be drawn is determined based on theestimated position of the particular part and a shape of a body surfaceof the user.

Appendix 25

The program according to Appendix 1, wherein the particular movement isa predetermined movement for measurement of the movement of the body ofthe user, and in the step of presenting, the avatar is presented to theuser each time the result of the sensing is obtained, the result of thesensing of the user at a predetermined timing at a breakpoint of theparticular movement is stored in a storage unit, and an avatar at thepredetermined timing arranged side by side with the avatar is presentedto the user.

Appendix 26

The program according to Appendix 1, wherein in the step of presentingthe avatar, the program makes the processor perform: a step ofestimating positions of at least three particular parts of the userbased on the result of the sensing of the movement of the body of theuser; and a step of determining a position of an object to be drawn thatindicates a positional relationship, with respect to any of the at leastthree particular parts, between others of the particular parts in ahorizontal plane to display the object at the determined position, andin the step of determining, the position of the figure object to bedrawn in association with the avatar is determined based on a result ofthe estimation of the positions of the at least three parts.

Appendix 27

A program that makes a computer operate, wherein the program makes aprocessor of the computer perform: a step of sequentially sensing amovement of a body of a user; a step of estimating positions of at leastthree particular parts of the user based on a result of the sensing ofthe movement of the body of the user; and a step of determining aposition of a figure object to be drawn that indicates a positionalrelationship, with respect to any of the at least three particular partsas a reference, between others of the particular parts in a horizontalplane to display the figure object at the determined position.

Appendix 28

The program according to Appendix 27, wherein in the step of displayingthe figure object, figure objects representing particular parts that arenot the reference are displayed as figure objects having a same shape orsimilar shapes.

Appendix 29

A method of making a computer operate, wherein a processor of thecomputer performs: a step of sequentially sensing a movement of a bodyof a user; a step of presenting, to the user, an avatar corresponding tothe user, rather than a photographed image of an appearance of the user,based on a result of the sensing so that the user perceives a shape of asurface of the body of the user; a step of determining a position of afigure object to be drawn that serves as a guide for movement of thebody of the user in association with the avatar based on the result ofthe sensing; and a step of displaying the figure object at thedetermined position.

Appendix 30

An information processing apparatus, wherein a control unit of theinformation processing apparatus performs: a step of sequentiallysensing a movement of a body of a user; a step of presenting, to theuser, an avatar corresponding to the user, rather than a photographedimage of an appearance of the user, based on a result of the sensing sothat the user perceives a shape of a surface of the body of the user; astep of determining a position of a figure object to be drawn thatserves as a guide for movement of the body of the user in associationwith the avatar based on the result of the sensing; and a step ofdisplaying the figure object at the determined position.

REFERENCE SIGNS LIST

-   10 terminal apparatus-   20 server-   30 measurement unit-   31 measurement apparatus-   32 mat-   33 pole-   132 display-   181 user information-   182 measurement item information-   183 measurement history information-   184 friend list-   185 training item information-   281 user information database-   282 measurement item information-   283 measurement history database-   284 friend list-   285 training item information-   287 measurement result database-   288 evaluation result database-   289 user questionnaire information-   290 evaluation result database-   2601 trainee information database-   2602 trainer information database-   2603 training history database

1. An information processing apparatus comprising a processor and amemory with instructions thereon, wherein the instructions uponexecution by the processor, cause the processor to: sequentially sense amovement of a body of a user; present, to the user, an avatarcorresponding to the user, rather than a photographed image of anappearance of the user, based on a result of the sensing so that theuser perceives a shape of a surface of the body of the user; determine aposition of a figure object to be drawn that serves as a guide formovement of the body of the user in association with the avatar based onthe result of the sensing; and display the figure object at thedetermined position.
 2. The apparatus according to claim 1, wherein inthe processor presents the avatar to the user with a reduced amount ofinformation compared with the photographed image.
 3. The apparatusaccording to claim 1, wherein the processor presents the avatar to theuser each time the result of the sensing is obtained.
 4. The apparatusaccording to claim 1, wherein in the processor presents the avatar suchthat an avatar corresponding to the user viewed in a first direction andan avatar corresponding to the user viewed in a second direction that isdifferent from the first direction are drawn side by side and presentedto the user.
 5. The apparatus according to claim 1, wherein theprocessor senses the movement of the body of the user as point clouddata, and the processor presents the avatar such that the avatar isdrawn as a set of graphic elements based on point cloud data andpresented to the user.
 6. The apparatus according to claim 1, whereinthe instructions upon execution by the processor, cause the processorto: estimate positions of a plurality of particular parts of the userbased on the result of the sensing of the movement of the body of theuser; and draw the avatar while not drawing a result of the estimationof the positions of the particular parts, and evaluate a way of movementof the body of the user based on the result of the estimation of thepositions of the particular parts.
 7. The apparatus according to claim1, wherein the processor presents the avatar such that the avatar drawnis updated in response to the result of the sensing of the movement ofthe body of the user, and the processor displays the figure object at afixed position regardless of the movement of the body of the user. 8.The apparatus according to claim 7, wherein the instructions uponexecution by the processor, cause the processor to: estimate a positionof a particular part of the user based on the result of the sensing ofthe movement of the body of the user; and draw, along with the avatar, atime-series history of positions of a first part of the user obtained bythe sensing, the first part being a subject of the measurement of themovement of the body of the user.
 9. The apparatus according to claim 7,wherein the processor determines to draw a linear figure object inparallel with a body axis of the user at a fixed position regardless ofthe movement of the body of the user based on the result of the sensingof the user.
 10. The apparatus according to claim 1, wherein theprocessor determines the position of the figure object to be drawn basedon a result of sensing of a first posture of the user before starting aparticular movement, the figure object serving as a guide for the userto make the particular movement.
 11. The apparatus according to claim10, wherein the particular movement is a predetermined movement formeasurement of the movement of the body of the user, and the processordetermines the position of the figure object to be drawn based on thefirst posture, the figure object serving as a guide for movement of thebody of the user at a predetermined timing in the particular movement.12. The apparatus according to claim 10, wherein the processordetermines the position of the figure object to be drawn based on thefirst posture, the figure object serving as a guide for at least one ofa group consisting of a front-rear balance and a left-right balance ofthe body of the user.
 13. The apparatus according to claim 10, whereinthe processor determines the position of the figure object to be drawnbased on the first posture, the figure object serving as a guide for asecond posture that can be assumed by the user while the user is makingthe particular movement.
 14. The apparatus according to claim 13,wherein the particular movement is a movement made by the user in astate where the user has taken a big step forward, and the processordetermines the position of the figure object to be drawn based on thefirst posture, the figure object serving as a guide for a posture of theuser in which the user has stepped a front foot forward.
 15. Theapparatus according to claim 14, wherein the instructions upon executionby the processor, cause the processor to estimate a position of a lowerlimb of the user based on the result of the sensing of the movement ofthe body of the user, and the processor determines the position of thefigure object to be drawn based on the result of the estimation, thefigure object serving as a guide for a step position of the user havingtaken a big step forward.
 16. The apparatus according to claim 13,wherein the instructions upon execution by the processor, cause theprocessor to: estimate a position of a particular part of the user basedon the result of the sensing of the movement of the body of the user;and evaluate whether the user is properly assuming the second posturebased on a positional relationship between the estimated position of theparticular part and the position of the figure object.
 17. The apparatusaccording to claim 16, wherein when the processor evaluates that theuser is properly assuming the second posture, the processor presentsthat the user is properly assuming the second posture to the user. 18.The apparatus according to claim 16, wherein the particular movementinvolves a body movement made by the user in the second posture, and inresponse to evaluation that the user is properly assuming the secondposture, the processor starts measurement of the particular movement,and presents the start of the measurement to the user.
 19. The apparatusaccording to claim 1, wherein the instructions upon execution by theprocessor, cause the processor to estimate positions of a plurality ofparticular parts of the user based on the result of the sensing of themovement of the body of the user, and the processor determines, aposition of a linear figure object to be drawn based on the estimatedpositions of the particular parts, the linear figure object allowing theuser to perceive whether the particular parts are properly moving in themovement of the body of the user.
 20. The apparatus according to claim19, wherein the processor displays a line segment as the figure objectalong with the avatar, the line segment being an extension of a lineextending from a peripheral part of a particular part to the particularpart.
 21. The apparatus according to claim 20, wherein the processordraws and displays a figure object having a large size compared with awidth of the line segment at an end point of the line segment.
 22. Theapparatus according to claim 21, wherein the processor displays thefigure object corresponding to the end point of the line segment in sucha manner that the figure object moves in association with the result ofthe sensing of the movement of the body of the user, and a way ofdisplay of the figure object is changed based on a positionalrelationship between the figure object arranged in a fixed mannerregardless of the movement of the body of the user and the figure objectcorresponding to the end point.
 23. The apparatus according to claim 19,wherein the instructions upon execution by the processor, cause theprocessor to estimate positions of a plurality of particular parts ofthe user based on the result of the sensing of the movement of the user,and the processor draws and displays a figure object indicating aninclination of a first part of the user and a figure object indicatingan inclination of a second part that is different from the first part inassociation with the first part and the second part, respectively. 24.The apparatus according to claim 19, wherein the instructions uponexecution by the processor, cause the processor to estimate a positionof a particular part of the user based on the result of the sensing ofthe movement of the body of the user, and the processor determines theposition of the figure object to be drawn based on the estimatedposition of the particular part and a shape of a body surface of theuser.
 25. The apparatus according to claim 1, wherein the particularmovement is a predetermined movement for measurement of the movement ofthe body of the user, and the processor presents the avatar to the usereach time the result of the sensing is obtained, the result of thesensing of the user at a predetermined timing at a breakpoint of theparticular movement is stored in a storage unit, and an avatar at thepredetermined timing arranged side by side with the avatar is presentedto the user.
 26. The apparatus according to claim 1, wherein theinstructions upon execution by the processor, cause the processor to:estimate positions of at least three particular parts of the user basedon the result of the sensing of the movement of the body of the user;and determine a position of an object to be drawn that indicates apositional relationship, with respect to any of the at least threeparticular parts, between others of the particular parts in a horizontalplane to display the object at the determined position, and theprocessor determines the position of the figure object to be drawn inassociation with the avatar based on a result of the estimation of thepositions of the at least three parts.
 27. An information processingapparatus comprising a processor and a memory with instructions thereon,wherein the instructions upon execution by the processor, cause theprocessor to: sequentially sense a movement of a body of a user;estimate positions of at least three particular parts of the user basedon a result of the sensing of the movement of the body of the user; anddetermine a position of a figure object to be drawn that indicates apositional relationship, with respect to any of the at least threeparticular parts as a reference, between others of the particular partsin a horizontal plane to display the figure object at the determinedposition.
 28. The apparatus according to claim 27, wherein the processordisplays figure objects representing particular parts that are not thereference as figure objects having a same shape or similar shapes.
 29. Amethod of making a computer execute: a step of sequentially sensing amovement of a body of a user; a step of presenting, to the user, anavatar corresponding to the user, rather than a photographed image of anappearance of the user, based on a result of the sensing so that theuser perceives a shape of a surface of the body of the user; a step ofdetermining a position of a figure object to be drawn that serves as aguide for movement of the body of the user in association with theavatar based on the result of the sensing; and a step of displaying thefigure object at the determined position.